Oct 31 00:30:19.932300 ip-10-0-142-217 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 31 00:30:19.932311 ip-10-0-142-217 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 31 00:30:19.932318 ip-10-0-142-217 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 31 00:30:19.932588 ip-10-0-142-217 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 31 00:30:30.068742 ip-10-0-142-217 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 31 00:30:30.068757 ip-10-0-142-217 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4545dc03075748d9ad7d8da780e5676c --
Oct 31 00:32:37.124523 ip-10-0-142-217 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 00:32:37.493555 ip-10-0-142-217 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:32:37.493555 ip-10-0-142-217 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 00:32:37.493555 ip-10-0-142-217 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:32:37.493555 ip-10-0-142-217 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:32:37.501238 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501149    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.501238 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501239    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501244    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501250    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501253    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501257    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501260    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501262    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501265    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501268    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501270    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501273    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501276    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501278    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501281    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501284    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501286    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501289    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501291    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501295    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.501313 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501299    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501303    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501306    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501309    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501312    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501315    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501340    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501343    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501346    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501348    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501351    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501353    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501356    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501359    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501361    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501363    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501366    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501368    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501371    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.501748 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501373    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501375    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501378    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501380    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501383    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501385    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501388    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501390    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501393    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501395    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501398    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501400    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501402    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501405    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501410    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501412    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501415    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501418    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501420    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.502218 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501422    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501426    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501428    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501431    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501433    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501436    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501439    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501441    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501444    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501446    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501448    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501451    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501453    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501455    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501458    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501461    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501463    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501465    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501468    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501470    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.502669 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501474    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501476    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501479    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501481    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501483    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501486    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501490    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501493    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501495    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501498    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501500    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501503    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501505    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501509    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501512    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501514    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501517    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501519    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501522    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501524    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.503116 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501526    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501529    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501531    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501534    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501536    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501538    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501541    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.501543    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502307    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502315    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502318    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502321    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502323    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502327    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502330    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502332    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502335    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502338    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502341    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.503581 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502344    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502347    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502349    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502352    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502355    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502357    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502360    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502362    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502365    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502367    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502369    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502372    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502375    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502377    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502379    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502382    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502384    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502386    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502389    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502391    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.504010 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502394    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502398    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502401    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502404    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502406    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502409    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502411    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502417    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502420    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502422    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502426    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502429    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502431    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502434    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502436    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502439    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502441    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502444    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502446    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.504530 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502448    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502451    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502453    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502456    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502458    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502461    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502463    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502465    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502468    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502470    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502473    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502475    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502477    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502480    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502482    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502484    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502487    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502490    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502492    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502495    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.504977 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502497    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502505    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502508    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502511    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502514    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502517    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502520    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502522    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502524    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502527    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502529    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502532    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502534    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502537    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502539    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502542    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502544    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502547    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502549    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.505463 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502551    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502554    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502557    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502559    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502561    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502564    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502566    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502569    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502573    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502577    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502579    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502582    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502584    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502587    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502589    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502597    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.502601    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503110    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503115    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503131    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.505916 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503134    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503137    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503140    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503143    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503145    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503148    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503151    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503153    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503156    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503158    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503161    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503163    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503166    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503168    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503170    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503173    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503175    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503178    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503180    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503183    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.506389 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503185    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503188    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503191    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503193    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503195    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503198    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503200    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503209    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503213    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503215    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503218    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503220    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503223    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503236    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503239    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503242    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503244    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503247    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503249    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503251    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.506842 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503254    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503256    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503259    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503263    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503267    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503270    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503273    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503275    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503278    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503280    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503283    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503285    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503288    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503290    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503293    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503295    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503298    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503301    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503304    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.507363 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503306    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503314    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503317    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503320    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503322    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503325    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503327    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503329    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503332    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503334    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503341    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503345    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503347    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503350    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503352    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503355    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503357    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503359    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503362    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.507790 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503365    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503368    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503370    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503376    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503378    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503381    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503383    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503385    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503387    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503390    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503392    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503395    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503398    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503400    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503403    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503412    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503415    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503417    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503419    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.508262 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503422    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503424    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503427    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503429    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503432    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.503435    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504520    2578 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504530    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504540    2578 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504545    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504550    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504553    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504558    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504563    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504566    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504569    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504573    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504576    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504579    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504582    2578 flags.go:64] FLAG: --cgroup-root=""
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504585    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504588    2578 flags.go:64] FLAG: --client-ca-file=""
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504591    2578 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 00:32:37.508732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504594    2578 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504602    2578 flags.go:64] FLAG: --cluster-domain=""
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504605    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504608    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504612    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504615    2578 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504625    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504628    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504631    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504634    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504637    2578 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504641    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504644    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504647    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504650    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504655    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504658    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504661    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504663    2578 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504666    2578 flags.go:64] FLAG: --enable-server="true"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504669    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504679    2578 flags.go:64] FLAG: --event-burst="100"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504682    2578 flags.go:64] FLAG: --event-qps="50"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504685    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 00:32:37.509284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504688    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504692    2578 flags.go:64] FLAG: --eviction-hard=""
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504696    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504699    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504702    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504705    2578 flags.go:64] FLAG: --eviction-soft=""
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504708    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504717    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504720    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504723    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504726    2578 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504729    2578 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504732    2578 flags.go:64] FLAG: --feature-gates=""
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504736    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504739    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504748    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504751    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504754    2578 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504757    2578 flags.go:64] FLAG: --help="false"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504760    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504763    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504766    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504769    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 00:32:37.509855 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504772    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504775    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504778    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504781    2578 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504784    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504786    2578 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504789    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504792    2578 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504795    2578 flags.go:64] FLAG: --kube-reserved=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504798    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504801    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504804    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504807    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504810    2578 flags.go:64] FLAG: --lock-file=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504812    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504815    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504820    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504825    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504828    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504831    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504834    2578 flags.go:64] FLAG: --logging-format="text"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504837    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504841    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504843    2578 flags.go:64] FLAG: --manifest-url=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504846    2578 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 00:32:37.510385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504856    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504860    2578 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504863    2578 flags.go:64] FLAG: --max-pods="110"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504867    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504870    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504873    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504876    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504878    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504882    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504885    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504892    2578 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504895    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504898    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504901    2578 flags.go:64] FLAG: --pod-cidr=""
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504904    2578 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504907    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504910    2578 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504913    2578 flags.go:64] FLAG: --port="10250"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504916    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504919    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-05521f2a842664096"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504922    2578 flags.go:64] FLAG: --qos-reserved=""
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504925    2578 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504928    2578 flags.go:64] FLAG: --register-node="true"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504934    2578 flags.go:64] FLAG: --register-with-taints=""
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504938    2578 flags.go:64] FLAG: --registry-burst="10"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504941    2578 flags.go:64] FLAG: --registry-qps="5"
Oct 31 00:32:37.511038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504943    2578 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504946    2578 flags.go:64] FLAG: --reserved-memory=""
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504950    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504953    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504956    2578 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504959    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504962    2578 flags.go:64] FLAG: --runonce="false"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504966    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504970    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504973    2578 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504976    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504979    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504982    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504986    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504989    2578 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504992    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504995    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.504998    2578 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505001    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505004    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505007    2578 flags.go:64] FLAG: --system-cgroups=""
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505010    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505015    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505018    2578 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505021    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 00:32:37.511687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505027    2578 flags.go:64] FLAG: --tls-min-version=""
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505030    2578 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505033    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505036    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505040    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505044    2578 flags.go:64] FLAG: --v="2"
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505048    2578 flags.go:64] FLAG: --version="false"
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505055    2578 flags.go:64] FLAG: --vmodule=""
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505059    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505063    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505221    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505236    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505240    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505242    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505245    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505250    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505253    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505256    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505258    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505261    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505264    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505266    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.512336 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505269    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505272    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505274    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505277    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505280    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505282    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505285    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505287    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505290    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505293    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505295    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505298    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505300    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505303    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505306    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505309    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505312    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505315    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505317    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505320    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.512853 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505322    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505325    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505340    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505342    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505345    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505348    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505352    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505354    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505357    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505359    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505361    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505364    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505367    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505369    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505371    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505374    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505377    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505379    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505382    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505384    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.513369 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505386    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505389    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505392    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505394    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505397    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505399    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505403    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505405    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505408    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505412    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505415    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505418    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505421    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505424    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505427    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505429    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505432    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505436    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.513838 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505438    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505442    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505446    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505449    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505452    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505454    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505457    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505459    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505462    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505464    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505467    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505470    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505472    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505475    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505478    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505480    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505483    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505486    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505488    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.514271 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505491    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505494    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505497    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505500    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505502    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505505    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505507    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505510    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505512    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505515    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505518    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505520    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505524    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505527    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505530    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505532    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.514708 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505535    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.505561    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505612    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505617    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505620    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505623    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505626    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505629    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505632    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505634    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505637    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505639    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505642    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505644    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505647    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505649    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.515076 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505652    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505655    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505659    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505661    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505664    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505666    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505669    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505671    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505674    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505676    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505678    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505681    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505685    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505687    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505690    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505692    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505696    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505699    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505701    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505704    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.515459 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505706    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505708    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505711    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505713    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505716    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505718    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505721    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505723    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505726    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505730    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505734    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505737    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505740    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505742    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505746    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505749    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505751    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505754    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.516204 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505756    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505759    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505762    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505765    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505769    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505771    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505777    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505780    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505783    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505785    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505789    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505792    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505794    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505797    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505800    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505802    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505805    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505808    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505810    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.516764 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505813    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505815    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505818    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505821    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505823    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505826    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505828    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505830    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505833    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505837    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505839    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505842    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505845    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505847    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505850    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505852    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505855    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505857    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505860    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505863    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.517215 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505866    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505868    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505870    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505873    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505875    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505878    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505881    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505883    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505886    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505888    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505891    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505893    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505897    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505899    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.505902    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.512933    2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.512949    2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.512966    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.512971    2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513031    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.517801 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513036    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513039    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513042    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513045    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513048    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513051    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513053    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513056    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513059    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513061    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513064    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513067    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513069    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513072    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513076    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513079    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513083    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513086    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513088    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513091    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.518594 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513094    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513097    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513099    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513102    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513105    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513107    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513110    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513113    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513117    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513119    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513122    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513125    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513127    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513130    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513133    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513135    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513138    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513141    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513143    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513146    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.519192 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513149    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513151    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513154    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513156    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513159    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513161    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513164    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513167    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513169    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513172    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513174    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513177    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513179    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513182    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513184    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513187    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513189    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513191    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513194    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.519952 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513197    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513200    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513203    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513206    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513208    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513211    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513213    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513216    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513219    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513222    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513239    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513244    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513247    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513250    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513253    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513255    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513258    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513261    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513264    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513266    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.520440 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513269    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513272    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513275    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513277    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513280    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513282    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513285    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513289    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513292    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513295    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513298    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513301    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513304    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513307    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513309    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513313    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513316    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513318    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513321    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.520948 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513323    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513326    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513341    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513344    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513346    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513349    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513351    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.513380    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513521    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513526    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513529    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513532    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513535    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513537    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513540    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513543    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.521417 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513545    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513548    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513550    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513553    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513555    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513558    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513560    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513563    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513565    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513568    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513570    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513573    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513575    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513579    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513581    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513584    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513586    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513588    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513591    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.521871 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513613    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513616    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513619    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513621    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513624    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513626    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513629    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513631    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513634    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513637    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513639    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513642    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513644    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513647    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513649    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513651    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513654    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513657    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513659    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.522355 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513661    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513664    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513666    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513669    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513672    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513674    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513677    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513679    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513682    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513686    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513690    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513693    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513696    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513698    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513702    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513705    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513708    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513710    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.522787 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513713    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513715    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513718    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513721    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513723    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513726    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513729    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513731    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513733    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513736    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513738    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513740    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513743    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513745    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513748    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513750    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513752    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513755    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513757    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513760    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.523439 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513762    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513764    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513767    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513770    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513772    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513775    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513777    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513779    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513782    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513784    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513786    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513789    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513791    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513794    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513796    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513799    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513801    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513803    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513806    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513808    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.524330 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513810    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513813    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.513834    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513863    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513867    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513870    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513873    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513875    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513878    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513880    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513883    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513885    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513889    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513893    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513896    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:32:37.525172 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513900    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513907    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513911    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513914    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513917    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513920    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513922    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513924    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513927    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513929    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513932    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513935    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513937    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513940    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513942    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513944    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513947    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513949    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513952    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:32:37.525619 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513954    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513957    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513959    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513961    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513964    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513966    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513969    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513971    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513974    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513976    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513979    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513981    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513983    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513986    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513988    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513991    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513994    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513996    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.513998    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514001    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:32:37.526057 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514003    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514006    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514008    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514011    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514013    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514015    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514018    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514020    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514022    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514025    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514028    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514030    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514033    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514036    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514038    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514043    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514046    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514048    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514051    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514053    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:32:37.526574 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514055    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514058    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514061    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514063    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514065    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514068    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514071    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514073    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514076    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514079    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514082    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514084    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514087    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514089    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514091    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514094    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514096    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514098    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514102    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:32:37.527015 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514104    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514107    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514109    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514111    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514114    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514116    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514119    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514121    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514125    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514127    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514130    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514132    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514135    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514137    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514140    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:37.514142    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.514195    2578 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.519626    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.520778    2578 server.go:1014] "Starting client certificate rotation"
Oct 31 00:32:37.527554 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.520873    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:32:37.527982 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.521709    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:32:37.527982 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.523543    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:32:37.527982 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.525260    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:32:37.538428 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.538409    2578 log.go:25] "Validated CRI v1 runtime API"
Oct 31 00:32:37.542331 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.542318    2578 log.go:25] "Validated CRI v1 image API"
Oct 31 00:32:37.543097 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.543073    2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 00:32:37.549242 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.549206    2578 fs.go:135] Filesystem UUIDs: map[08ca7b1e-a9f1-45f9-9e06-617f03eaae71:/dev/nvme0n1p3 77b26557-0233-42d6-aa53-6281f9eca005:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 31 00:32:37.549328 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.549243    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 00:32:37.552468 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.552448    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 00:32:37.554597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.554484    2578 manager.go:217] Machine: {Timestamp:2026-10-31 00:32:37.553001234 +0000 UTC m=+0.330255441 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101364 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24efd0f0ef3856c608c1c635d066b8 SystemUUID:ec24efd0-f0ef-3856-c608-c1c635d066b8 BootID:4545dc03-0757-48d9-ad7d-8da780e5676c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c3:6e:a1:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c3:6e:a1:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:19:e6:84:39:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 00:32:37.554597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.554594    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 00:32:37.554767 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.554699    2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 00:32:37.558124 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.558098    2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 00:32:37.558263 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.558125    2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-217.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 00:32:37.558312 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.558265    2578 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 00:32:37.558312 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.558274    2578 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 00:32:37.558312 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.558310    2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:32:37.558386 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.558324    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:32:37.559610 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.559600    2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 00:32:37.564208 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564196    2578 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 00:32:37.564347 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564337    2578 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 00:32:37.564377 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564368    2578 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 00:32:37.564409 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564377    2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 00:32:37.564409 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564388    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 00:32:37.564409 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564397    2578 kubelet.go:405] "Adding apiserver pod source"
Oct 31 00:32:37.564409 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564406    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 00:32:37.564732 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.564717    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:32:37.567555 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.567535    2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 00:32:37.574856 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.574841    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:32:37.575746 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575731    2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 00:32:37.575809 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575770    2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 00:32:37.575959 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575946    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 00:32:37.576046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575964    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 00:32:37.576046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575973    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 00:32:37.576046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575983    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 00:32:37.576046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.575991    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 00:32:37.576046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576001    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 00:32:37.576675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576662    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 00:32:37.576728 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576678    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 00:32:37.576728 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576695    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 00:32:37.576728 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576706    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 00:32:37.576728 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576725    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 00:32:37.576893 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.576738    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 00:32:37.577432 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.577421    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 00:32:37.577479 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.577436    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 00:32:37.581352 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.581326    2578 server.go:1263] "Started kubelet"
Oct 31 00:32:37.581432 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.581407    2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 00:32:37.581518 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.581480    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 00:32:37.581547 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.581535    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 00:32:37.582731 ip-10-0-142-217 systemd[1]: Started Kubernetes Kubelet.
Oct 31 00:32:37.583144 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.583127    2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 00:32:37.584057 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.584045    2578 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 00:32:37.589177 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.588029    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-217.us-east-2.compute.internal.18e375f1b36b2a89  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-217.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-217.us-east-2.compute.internal,},FirstTimestamp:2026-10-31 00:32:37.581302409 +0000 UTC m=+0.358556615,LastTimestamp:2026-10-31 00:32:37.581302409 +0000 UTC m=+0.358556615,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-217.us-east-2.compute.internal,}"
Oct 31 00:32:37.589796 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.589766    2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 00:32:37.590752 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.590734    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 00:32:37.592153 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.592117    2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 00:32:37.592276 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.592259    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:37.592717 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.592692    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9g4w7"
Oct 31 00:32:37.593003 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.592984    2578 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 00:32:37.593003 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593004    2578 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 00:32:37.593156 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593045    2578 factory.go:55] Registering systemd factory
Oct 31 00:32:37.593156 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593060    2578 factory.go:223] Registration of the systemd container factory successfully
Oct 31 00:32:37.593156 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.592991    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 00:32:37.593156 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593148    2578 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 00:32:37.593156 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593155    2578 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 00:32:37.593405 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593369    2578 factory.go:153] Registering CRI-O factory
Oct 31 00:32:37.593405 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593384    2578 factory.go:223] Registration of the crio container factory successfully
Oct 31 00:32:37.593506 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593435    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 00:32:37.593506 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593454    2578 factory.go:103] Registering Raw factory
Oct 31 00:32:37.593506 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.593467    2578 manager.go:1199] Started watching for new ooms in manager
Oct 31 00:32:37.594522 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.594507    2578 manager.go:319] Starting recovery of all containers
Oct 31 00:32:37.598670 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.598547    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-217.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 00:32:37.598830 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.598802    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 00:32:37.598963 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.598940    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-217.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 00:32:37.599586 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.599557    2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-217.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 00:32:37.602127 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.602105    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9g4w7"
Oct 31 00:32:37.602331 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.602307    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 31 00:32:37.608389 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.606972    2578 manager.go:324] Recovery completed
Oct 31 00:32:37.611357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.611345    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.613014 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.612998    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.613081 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.613026    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.613081 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.613036    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.613613 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.613592    2578 cpu_manager.go:226] "Starting" policy="none"
Oct 31 00:32:37.613613 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.613607    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.613753 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.613609    2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 00:32:37.613753 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.613641    2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 00:32:37.614255 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.614224    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.614363 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.614268    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.614363 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.614280    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.615109 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.615042    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-217.us-east-2.compute.internal.18e375f1b54f0821  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-217.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-217.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-217.us-east-2.compute.internal,},FirstTimestamp:2026-10-31 00:32:37.613013025 +0000 UTC m=+0.390267231,LastTimestamp:2026-10-31 00:32:37.613013025 +0000 UTC m=+0.390267231,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-217.us-east-2.compute.internal,}"
Oct 31 00:32:37.615614 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.615603    2578 policy_none.go:50] "Start"
Oct 31 00:32:37.615646 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.615618    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 00:32:37.615646 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.615627    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 00:32:37.617667 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.617657    2578 policy_none.go:44] "Start"
Oct 31 00:32:37.679204 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.673881    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 00:32:37.679204 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.674990    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 00:32:37.679204 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.675017    2578 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 00:32:37.679204 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.675038    2578 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 00:32:37.679204 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.675076    2578 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 00:32:37.679204 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.678077    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:32:37.692622 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.692604    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:37.775164 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.775140    2578 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 31 00:32:37.791785 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.791754    2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 00:32:37.793026 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.793011    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:37.794193 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.794180    2578 manager.go:341] "Starting Device Plugin manager"
Oct 31 00:32:37.795055 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.795031    2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 00:32:37.795055 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.795057    2578 server.go:87] "Starting device plugin registration server"
Oct 31 00:32:37.795378 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.795364    2578 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 00:32:37.795438 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.795381    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 00:32:37.795490 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.795445    2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 00:32:37.795538 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.795512    2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 00:32:37.795538 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.795520    2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 00:32:37.796018 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.795999    2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 00:32:37.796076 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.796053    2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:37.805530 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.805514    2578 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-217.us-east-2.compute.internal\" not found" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.896000 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.895980    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.897500 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.897483    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.897574 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.897515    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.897574 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.897528    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.897574 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.897552    2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.907199 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.907182    2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.907254 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.907203    2578 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-217.us-east-2.compute.internal\": node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:37.925930 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:37.925912    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:37.976026 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.976000    2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"]
Oct 31 00:32:37.976136 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.976066    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.976958 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.976938    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.977036 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.976969    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.977036 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.976979    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.979237 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979214    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.979401 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979386    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.979437 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979415    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.979948 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979929    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.980020 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979949    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.980020 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979972    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.980020 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979983    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.980138 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.979955    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.980138 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.980049    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.982207 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.982193    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.982286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.982217    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:32:37.982975 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.982960    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:32:37.983038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.982992    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:32:37.983038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.983006    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:32:37.995620 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.995605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.995698 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.995626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:37.995698 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:37.995642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ece4c01c041ef372ae3784c8037bb134-config\") pod \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"ece4c01c041ef372ae3784c8037bb134\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.013783 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:38.013765    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.016402 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:38.016388    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.026679 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:38.026665    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:38.096306 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:38.096275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.096306 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:38.096307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.096473 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:38.096329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ece4c01c041ef372ae3784c8037bb134-config\") pod \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"ece4c01c041ef372ae3784c8037bb134\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.096473 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:38.096363    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.096473 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:38.096387    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ece4c01c041ef372ae3784c8037bb134-config\") pod \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"ece4c01c041ef372ae3784c8037bb134\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.096473 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:38.096390    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:38.127402 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:38.127375    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:39.973460 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:39.973435    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:40.065010 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.064926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:40.067073 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.067059    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:40.074513 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.074395    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:40.174961 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.174933    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:32:40.259843 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.259818    2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:32:40.265872 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.265848    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 00:32:40.265986 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.265959    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 00:32:40.266027 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.265999    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 00:32:40.310205 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.310178    2578 apiserver.go:52] "Watching apiserver"
Oct 31 00:32:40.331974 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.331912    2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 00:32:40.332283 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.332264    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp","openshift-cluster-node-tuning-operator/tuned-x2sfq","openshift-dns/node-resolver-gbgzq","openshift-multus/multus-additional-cni-plugins-9tdv7","openshift-multus/network-metrics-daemon-5659t","kube-system/konnectivity-agent-hrtzv","openshift-multus/multus-f4ddz","openshift-ovn-kubernetes/ovnkube-node-nmwrk","openshift-network-diagnostics/network-check-target-kwth8","openshift-network-operator/iptables-alerter-d66sh","openshift-image-registry/node-ca-6hznk","kube-system/global-pull-secret-syncer-6gbrr"]
Oct 31 00:32:40.334804 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.334791    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.336731 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.336714    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 00:32:40.336842 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.336826    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.339284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.337843    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:40.339397 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.339330    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.339503 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.339482    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kxg2f\""
Oct 31 00:32:40.339503 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.339486    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 00:32:40.339649 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.339554    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.339968 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.339946    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.340462 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.340443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.340564 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.340542    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jwzf5\""
Oct 31 00:32:40.340961 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.340948    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.343010 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.342996    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.344662 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344728 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-socket-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.344728 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnq72\" (UniqueName: \"kubernetes.io/projected/bb393179-7795-4aa8-bd36-538fe2821808-kube-api-access-rnq72\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.344807 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysconfig\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344807 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-tuned\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344863 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344807    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-tmp\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344863 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-registration-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.344863 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-device-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.344946 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-conf\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344946 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-host\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344946 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj6gj\" (UniqueName: \"kubernetes.io/projected/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-kube-api-access-cj6gj\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.344946 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15983cd6-0227-4191-8770-4fd87b6a52b2-hosts-file\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.344946 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.344946 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344941    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-sys-fs\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-modprobe-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-lib-modules\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.344994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-var-lib-kubelet\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15983cd6-0227-4191-8770-4fd87b6a52b2-tmp-dir\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-kubernetes\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-systemd\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345298 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-run\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345298 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-sys\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.345298 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck9d4\" (UniqueName: \"kubernetes.io/projected/15983cd6-0227-4191-8770-4fd87b6a52b2-kube-api-access-ck9d4\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.345298 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.345220    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:40.345298 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.345285    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:40.346014 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.346001    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ppkcp\""
Oct 31 00:32:40.347377 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.347356    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.349535 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.349522    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.351818 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.351805    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.353918 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.353871    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 00:27:39 +0000 UTC" deadline="2028-04-14 21:14:06.28798561 +0000 UTC"
Oct 31 00:32:40.353918 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.353916    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12764h41m25.934073061s"
Oct 31 00:32:40.354037 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.353984    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:40.354074 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.354051    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:40.355970 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.355954    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:32:40.356046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.355988    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 00:32:40.356046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.356029    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:32:40.356238 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.356225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.358476 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.358463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.360656 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.360641    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"]
Oct 31 00:32:40.360717 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.360700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.360769 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.360740    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:40.363687 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.363655    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:32:40.363786 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.363775    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"]
Oct 31 00:32:40.376827 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.376805    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9klmt"
Oct 31 00:32:40.382757 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.382734    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9klmt"
Oct 31 00:32:40.418192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.418173    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:32:40.439634 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439610    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 00:32:40.439765 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439704    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.439765 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439718    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 00:32:40.439765 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439725    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.439907 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439847    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 00:32:40.439907 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439864    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 00:32:40.439907 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.439871    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.440292 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.440278    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.440358 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.440297    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.440358 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.440303    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 00:32:40.440358 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.440308    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 00:32:40.440358 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.440319    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 00:32:40.441308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441290    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.441308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441289    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8fmps\""
Oct 31 00:32:40.441308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441305    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.441454 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441388    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 00:32:40.441454 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441391    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 00:32:40.441454 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441403    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.441938 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441923    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 00:32:40.441938 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.441937    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 00:32:40.443024 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443008    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k5rkn\""
Oct 31 00:32:40.443102 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443060    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 00:32:40.443102 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443074    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:32:40.443102 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443090    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 00:32:40.443424 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443406    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hkzgb\""
Oct 31 00:32:40.443424 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443417    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b29sj\""
Oct 31 00:32:40.443558 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443464    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lfknl\""
Oct 31 00:32:40.443558 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443487    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 00:32:40.443558 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.443489    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cvzdq\""
Oct 31 00:32:40.445300 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-kubernetes\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.445373 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-system-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.445373 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-daemon-config\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.445482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445404    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-kubernetes\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.445482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnq72\" (UniqueName: \"kubernetes.io/projected/bb393179-7795-4aa8-bd36-538fe2821808-kube-api-access-rnq72\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.445482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-tuned\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.445626 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445551    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-dbus\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.445626 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-system-cni-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.445737 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovn-node-metrics-cert\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.445737 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-conf\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.445737 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-bin\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.445889 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445779    2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 00:32:40.445889 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-script-lib\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.445889 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/edb0ea49-807e-4139-a8bc-516f3b88b720-iptables-alerter-script\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.445889 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445864    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-conf\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.445889 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edb0ea49-807e-4139-a8bc-516f3b88b720-host-slash\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.446055 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwzqt\" (UniqueName: \"kubernetes.io/projected/edb0ea49-807e-4139-a8bc-516f3b88b720-kube-api-access-kwzqt\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.446055 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-sys-fs\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.446055 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.445989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-modprobe-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446055 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446032    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-sys-fs\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446058    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-var-lib-kubelet\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446101    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-modprobe-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446138    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-var-lib-kubelet\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-cnibin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.446236 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-kubelet\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-slash\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-systemd\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-run\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446359    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-systemd\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck9d4\" (UniqueName: \"kubernetes.io/projected/15983cd6-0227-4191-8770-4fd87b6a52b2-kube-api-access-ck9d4\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-systemd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-serviceca\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.446530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446492    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-run\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446563    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-socket-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysconfig\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-tmp\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-binary-copy\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446681    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysconfig\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446699    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-socket-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-registration-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj6gj\" (UniqueName: \"kubernetes.io/projected/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-kube-api-access-cj6gj\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.446865 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446857    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-k8s-cni-cncf-io\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446885    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-registration-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-netns\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-bin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-conf-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.446991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-etc-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-lib-modules\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-env-overrides\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447116    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447141    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-lib-modules\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-sys\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-socket-dir-parent\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447198    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-sys\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.447357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447214    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-kubelet\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-config\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447325    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-cnibin\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-cni-binary-copy\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447415    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-systemd-units\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-var-lib-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6hhh\" (UniqueName: \"kubernetes.io/projected/03f1dde4-3764-4c15-821f-54ca5cd3b47f-kube-api-access-f6hhh\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-device-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447620    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-device-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15983cd6-0227-4191-8770-4fd87b6a52b2-hosts-file\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rct8\" (UniqueName: \"kubernetes.io/projected/fce6d408-267e-49f1-ac1e-68176943fbf9-kube-api-access-9rct8\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447668    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15983cd6-0227-4191-8770-4fd87b6a52b2-hosts-file\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-netd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/542f4323-b31e-4491-9b6e-02a1916798ab-agent-certs\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.448022 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/542f4323-b31e-4491-9b6e-02a1916798ab-konnectivity-ca\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15983cd6-0227-4191-8770-4fd87b6a52b2-tmp-dir\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-multus\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-hostroot\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-multus-certs\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-netns\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.447983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-log-socket\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgzq2\" (UniqueName: \"kubernetes.io/projected/9557d837-f593-4d62-ad04-07b2c0b77987-kube-api-access-hgzq2\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-etc-kubernetes\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448096    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15983cd6-0227-4191-8770-4fd87b6a52b2-tmp-dir\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-node-log\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpqbn\" (UniqueName: \"kubernetes.io/projected/6643e3d6-52ac-4e9d-af24-8a18f92b3748-kube-api-access-zpqbn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448298    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-host\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-host\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.448597 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448388    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9fnb\" (UniqueName: \"kubernetes.io/projected/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-kube-api-access-n9fnb\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.449212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448408    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-host\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.449212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-kubelet-config\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.449212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.448447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-ovn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.449480 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.449460    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-tuned\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.449869 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.449851    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-tmp\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.454155 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.454124    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnq72\" (UniqueName: \"kubernetes.io/projected/bb393179-7795-4aa8-bd36-538fe2821808-kube-api-access-rnq72\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.454573 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.454558    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck9d4\" (UniqueName: \"kubernetes.io/projected/15983cd6-0227-4191-8770-4fd87b6a52b2-kube-api-access-ck9d4\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.454957 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.454939    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj6gj\" (UniqueName: \"kubernetes.io/projected/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-kube-api-access-cj6gj\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.548647 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.548647 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-cnibin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-kubelet\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-slash\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548725    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-slash\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548762    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-cnibin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548764    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-kubelet\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-systemd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548804    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-serviceca\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548822    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-systemd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-binary-copy\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.548859 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-k8s-cni-cncf-io\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548915    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.548952    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-k8s-cni-cncf-io\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-netns\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-bin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-conf-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-etc-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549057    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-netns\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-env-overrides\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549096    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549112    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-bin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549108    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-conf-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549111    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-etc-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.549339 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549235    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-serviceca\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-socket-dir-parent\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-kubelet\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-config\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.549342    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549369    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-kubelet\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-cnibin\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549392    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-socket-dir-parent\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-cni-binary-copy\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549423    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-cnibin\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-systemd-units\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-var-lib-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549491    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-binary-copy\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6hhh\" (UniqueName: \"kubernetes.io/projected/03f1dde4-3764-4c15-821f-54ca5cd3b47f-kube-api-access-f6hhh\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549514    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-env-overrides\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550139 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rct8\" (UniqueName: \"kubernetes.io/projected/fce6d408-267e-49f1-ac1e-68176943fbf9-kube-api-access-9rct8\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-netd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549582    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-systemd-units\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/542f4323-b31e-4491-9b6e-02a1916798ab-agent-certs\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549659    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-netd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.549607    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549670    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549660    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-var-lib-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.549658    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:32:41.049621234 +0000 UTC m=+2.081358332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/542f4323-b31e-4491-9b6e-02a1916798ab-konnectivity-ca\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-multus\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-hostroot\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549812    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.549843    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret podName:4c562d34-6b68-4800-a5a6-5c9e204a6050 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:41.049827738 +0000 UTC m=+2.081564833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret") pod "global-pull-secret-syncer-6gbrr" (UID: "4c562d34-6b68-4800-a5a6-5c9e204a6050") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549860    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-multus\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-multus-certs\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.550995 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549871    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-hostroot\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549896    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-multus-certs\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-netns\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-log-socket\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgzq2\" (UniqueName: \"kubernetes.io/projected/9557d837-f593-4d62-ad04-07b2c0b77987-kube-api-access-hgzq2\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.549992    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-netns\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550003    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-log-socket\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-etc-kubernetes\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550034    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-config\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550042    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-etc-kubernetes\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550010    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-cni-binary-copy\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-node-log\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqbn\" (UniqueName: \"kubernetes.io/projected/6643e3d6-52ac-4e9d-af24-8a18f92b3748-kube-api-access-zpqbn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-host\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550144    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-node-log\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9fnb\" (UniqueName: \"kubernetes.io/projected/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-kube-api-access-n9fnb\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550188    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.551709 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-kubelet-config\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550217    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-host\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-ovn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550256    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-kubelet-config\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550255    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-system-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550276    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/542f4323-b31e-4491-9b6e-02a1916798ab-konnectivity-ca\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-daemon-config\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-dbus\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550304    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-system-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550329    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-ovn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-system-cni-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovn-node-metrics-cert\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550367    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-dbus\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550370    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-system-cni-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-bin\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-script-lib\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550425    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-bin\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/edb0ea49-807e-4139-a8bc-516f3b88b720-iptables-alerter-script\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.552308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edb0ea49-807e-4139-a8bc-516f3b88b720-host-slash\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwzqt\" (UniqueName: \"kubernetes.io/projected/edb0ea49-807e-4139-a8bc-516f3b88b720-kube-api-access-kwzqt\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.550520    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edb0ea49-807e-4139-a8bc-516f3b88b720-host-slash\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.551430    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-daemon-config\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.551492    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.551516    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/edb0ea49-807e-4139-a8bc-516f3b88b720-iptables-alerter-script\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.551516    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-script-lib\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.552793 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.552389    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/542f4323-b31e-4491-9b6e-02a1916798ab-agent-certs\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.553034 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.552881    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovn-node-metrics-cert\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.561268 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.561248    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:32:40.561268 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.561268    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:32:40.561428 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.561278    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xglkq for pod openshift-network-diagnostics/network-check-target-kwth8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:40.561428 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:40.561323    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq podName:007129ea-79e7-4781-a23c-8491d433c8a9 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:41.061313995 +0000 UTC m=+2.093051117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xglkq" (UniqueName: "kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq") pod "network-check-target-kwth8" (UID: "007129ea-79e7-4781-a23c-8491d433c8a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:40.563372 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.563315    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rct8\" (UniqueName: \"kubernetes.io/projected/fce6d408-267e-49f1-ac1e-68176943fbf9-kube-api-access-9rct8\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.563483 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.563448    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6hhh\" (UniqueName: \"kubernetes.io/projected/03f1dde4-3764-4c15-821f-54ca5cd3b47f-kube-api-access-f6hhh\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:40.564850 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.564827    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqbn\" (UniqueName: \"kubernetes.io/projected/6643e3d6-52ac-4e9d-af24-8a18f92b3748-kube-api-access-zpqbn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.564927 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.564864    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwzqt\" (UniqueName: \"kubernetes.io/projected/edb0ea49-807e-4139-a8bc-516f3b88b720-kube-api-access-kwzqt\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.565239 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.565224    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9fnb\" (UniqueName: \"kubernetes.io/projected/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-kube-api-access-n9fnb\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.566119 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.566104    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgzq2\" (UniqueName: \"kubernetes.io/projected/9557d837-f593-4d62-ad04-07b2c0b77987-kube-api-access-hgzq2\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.636534 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.636471    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:32:40.674007 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.673981    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:32:40.685502 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.685476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:32:40.707644 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.707620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:32:40.741510 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.741476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:32:40.747271 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.747253    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:40.775918 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.775894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f4ddz"
Oct 31 00:32:40.808925 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.808905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:32:40.811049 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.811030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:32:40.815660 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.815643    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:32:40.819772 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:40.819733    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06019dffe47e9f0f78940dad30a48a0f.slice/crio-521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c WatchSource:0}: Error finding container 521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c: Status 404 returned error can't find the container with id 521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c
Oct 31 00:32:40.820419 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:40.820393    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece4c01c041ef372ae3784c8037bb134.slice/crio-ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348 WatchSource:0}: Error finding container ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348: Status 404 returned error can't find the container with id ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348
Oct 31 00:32:40.822220 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:40.822188    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb0ea49_807e_4139_a8bc_516f3b88b720.slice/crio-c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92 WatchSource:0}: Error finding container c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92: Status 404 returned error can't find the container with id c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92
Oct 31 00:32:40.825901 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:40.825882    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:32:40.826636 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:40.826617    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c7b1d82_f9da_4de8_bb8f_f8e76a938aff.slice/crio-64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e WatchSource:0}: Error finding container 64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e: Status 404 returned error can't find the container with id 64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e
Oct 31 00:32:40.827380 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:40.827359    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6643e3d6_52ac_4e9d_af24_8a18f92b3748.slice/crio-d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7 WatchSource:0}: Error finding container d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7: Status 404 returned error can't find the container with id d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7
Oct 31 00:32:41.053090 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.053004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:41.053090 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.053055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:41.053247 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.053154    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:41.053247 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.053193    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:41.053247 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.053217    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:32:42.053203527 +0000 UTC m=+3.084940630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:41.053247 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.053240    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret podName:4c562d34-6b68-4800-a5a6-5c9e204a6050 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:42.053228549 +0000 UTC m=+3.084965641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret") pod "global-pull-secret-syncer-6gbrr" (UID: "4c562d34-6b68-4800-a5a6-5c9e204a6050") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:41.085111 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:41.085085    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfde58a0_def5_41fb_8f2d_e825a8ab34fe.slice/crio-83d4fe095a2ab67cc15935637cbd61e6dc33b38540e7f8fd1923857e998c0109 WatchSource:0}: Error finding container 83d4fe095a2ab67cc15935637cbd61e6dc33b38540e7f8fd1923857e998c0109: Status 404 returned error can't find the container with id 83d4fe095a2ab67cc15935637cbd61e6dc33b38540e7f8fd1923857e998c0109
Oct 31 00:32:41.116455 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:41.116432    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb393179_7795_4aa8_bd36_538fe2821808.slice/crio-cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439 WatchSource:0}: Error finding container cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439: Status 404 returned error can't find the container with id cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439
Oct 31 00:32:41.153575 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.153543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:41.153766 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.153732    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:32:41.153812 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.153772    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:32:41.153812 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.153783    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xglkq for pod openshift-network-diagnostics/network-check-target-kwth8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:41.153876 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:41.153829    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq podName:007129ea-79e7-4781-a23c-8491d433c8a9 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:42.153816759 +0000 UTC m=+3.185553865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xglkq" (UniqueName: "kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq") pod "network-check-target-kwth8" (UID: "007129ea-79e7-4781-a23c-8491d433c8a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:41.233700 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:41.233671    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542f4323_b31e_4491_9b6e_02a1916798ab.slice/crio-9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344 WatchSource:0}: Error finding container 9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344: Status 404 returned error can't find the container with id 9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344
Oct 31 00:32:41.266395 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:41.266369    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15983cd6_0227_4191_8770_4fd87b6a52b2.slice/crio-fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e WatchSource:0}: Error finding container fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e: Status 404 returned error can't find the container with id fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e
Oct 31 00:32:41.273400 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:41.273376    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce6d408_267e_49f1_ac1e_68176943fbf9.slice/crio-b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6 WatchSource:0}: Error finding container b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6: Status 404 returned error can't find the container with id b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6
Oct 31 00:32:41.274077 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:32:41.274058    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9557d837_f593_4d62_ad04_07b2c0b77987.slice/crio-04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282 WatchSource:0}: Error finding container 04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282: Status 404 returned error can't find the container with id 04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282
Oct 31 00:32:41.385611 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.385507    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:27:40 +0000 UTC" deadline="2028-07-14 16:46:29.311553062 +0000 UTC"
Oct 31 00:32:41.385611 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.385558    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14944h13m47.925999633s"
Oct 31 00:32:41.425943 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.425893    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbgzq" event={"ID":"15983cd6-0227-4191-8770-4fd87b6a52b2","Type":"ContainerStarted","Data":"fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e"}
Oct 31 00:32:41.428408 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.428305    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hrtzv" event={"ID":"542f4323-b31e-4491-9b6e-02a1916798ab","Type":"ContainerStarted","Data":"9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344"}
Oct 31 00:32:41.430790 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.430764    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp" event={"ID":"bb393179-7795-4aa8-bd36-538fe2821808","Type":"ContainerStarted","Data":"cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439"}
Oct 31 00:32:41.433003 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.432981    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x2sfq" event={"ID":"dfde58a0-def5-41fb-8f2d-e825a8ab34fe","Type":"ContainerStarted","Data":"83d4fe095a2ab67cc15935637cbd61e6dc33b38540e7f8fd1923857e998c0109"}
Oct 31 00:32:41.435327 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.435306    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d66sh" event={"ID":"edb0ea49-807e-4139-a8bc-516f3b88b720","Type":"ContainerStarted","Data":"c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92"}
Oct 31 00:32:41.437507 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.437470    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"ece4c01c041ef372ae3784c8037bb134","Type":"ContainerStarted","Data":"ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348"}
Oct 31 00:32:41.441451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.441429    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerStarted","Data":"04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282"}
Oct 31 00:32:41.444262 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.444241    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f4ddz" event={"ID":"fce6d408-267e-49f1-ac1e-68176943fbf9","Type":"ContainerStarted","Data":"b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6"}
Oct 31 00:32:41.447360 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.447283    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7"}
Oct 31 00:32:41.459083 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.459055    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6hznk" event={"ID":"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff","Type":"ContainerStarted","Data":"64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e"}
Oct 31 00:32:41.461585 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.461562    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"06019dffe47e9f0f78940dad30a48a0f","Type":"ContainerStarted","Data":"521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c"}
Oct 31 00:32:41.643561 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:41.643483    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:32:42.061617 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.061502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:42.061617 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.061555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:42.061891 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.061712    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:42.061891 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.061790    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret podName:4c562d34-6b68-4800-a5a6-5c9e204a6050 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:44.061771534 +0000 UTC m=+5.093508628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret") pod "global-pull-secret-syncer-6gbrr" (UID: "4c562d34-6b68-4800-a5a6-5c9e204a6050") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:42.062215 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.062195    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:42.062316 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.062249    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:32:44.062231873 +0000 UTC m=+5.093968968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:42.162308 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.162271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:42.162514 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.162467    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:32:42.162514 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.162490    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:32:42.162514 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.162503    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xglkq for pod openshift-network-diagnostics/network-check-target-kwth8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:42.162699 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.162564    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq podName:007129ea-79e7-4781-a23c-8491d433c8a9 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:44.162544681 +0000 UTC m=+5.194281787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xglkq" (UniqueName: "kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq") pod "network-check-target-kwth8" (UID: "007129ea-79e7-4781-a23c-8491d433c8a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:42.386327 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.386230    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:27:40 +0000 UTC" deadline="2028-07-29 15:21:02.18995108 +0000 UTC"
Oct 31 00:32:42.386327 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.386273    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15302h48m19.80368266s"
Oct 31 00:32:42.421720 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.421684    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:42.421916 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.421841    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:42.421980 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.421944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:42.422046 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.422019    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:42.422102 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.422078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:42.422227 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:42.422143    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:42.669867 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:42.669370    2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:32:44.083965 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:44.083103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:44.083965 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:44.083150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:44.083965 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.083293    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:44.083965 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.083355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret podName:4c562d34-6b68-4800-a5a6-5c9e204a6050 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:48.083337674 +0000 UTC m=+9.115074789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret") pod "global-pull-secret-syncer-6gbrr" (UID: "4c562d34-6b68-4800-a5a6-5c9e204a6050") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:44.083965 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.083735    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:44.083965 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.083805    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:32:48.083790082 +0000 UTC m=+9.115527200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:44.184404 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:44.184293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:44.185663 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.184725    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:32:44.185663 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.184769    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:32:44.185663 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.184784    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xglkq for pod openshift-network-diagnostics/network-check-target-kwth8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:44.185663 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.184878    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq podName:007129ea-79e7-4781-a23c-8491d433c8a9 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:48.184835619 +0000 UTC m=+9.216572730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xglkq" (UniqueName: "kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq") pod "network-check-target-kwth8" (UID: "007129ea-79e7-4781-a23c-8491d433c8a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:44.422152 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:44.421147    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:44.422152 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.421284    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:44.422152 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:44.421926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:44.422152 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.422005    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:44.422152 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:44.422042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:44.422152 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:44.422091    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:45.511778 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:45.511239    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerStarted","Data":"d833e8c06b1002dc1a89966e507676a06ac371c9a86be6766e6db2b7883ae9c2"}
Oct 31 00:32:45.521720 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:45.520990    2578 generic.go:356] "Generic (PLEG): container finished" podID="06019dffe47e9f0f78940dad30a48a0f" containerID="4616211d79023616b9c5e2fbe43e9fb1c3efef1948246607f9982b0910606518" exitCode=0
Oct 31 00:32:45.521720 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:45.521070    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"06019dffe47e9f0f78940dad30a48a0f","Type":"ContainerDied","Data":"4616211d79023616b9c5e2fbe43e9fb1c3efef1948246607f9982b0910606518"}
Oct 31 00:32:46.421565 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.421524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:46.421785 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:46.421668    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:46.422090 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.422072    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:46.422197 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:46.422170    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:46.422263 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.422211    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:46.422314 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:46.422260    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:46.526435 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.526397    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"06019dffe47e9f0f78940dad30a48a0f","Type":"ContainerStarted","Data":"fe9e1a72e19b458f7820d4df15c673f7483b483174c5836b0da28589c181e02e"}
Oct 31 00:32:46.536490 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.536455    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hrtzv" event={"ID":"542f4323-b31e-4491-9b6e-02a1916798ab","Type":"ContainerStarted","Data":"aa459478088adef71a78c563ba625a29278c2207407a9b00de3f9c6fb7e87a34"}
Oct 31 00:32:46.543046 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.539451    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"ece4c01c041ef372ae3784c8037bb134","Type":"ContainerStarted","Data":"b96d92a7e6618c1ee9e0b3bfe82443fcf14f74406f0925301ec044acf875f2f8"}
Oct 31 00:32:46.544283 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.544255    2578 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="d833e8c06b1002dc1a89966e507676a06ac371c9a86be6766e6db2b7883ae9c2" exitCode=0
Oct 31 00:32:46.544399 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.544297    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerDied","Data":"d833e8c06b1002dc1a89966e507676a06ac371c9a86be6766e6db2b7883ae9c2"}
Oct 31 00:32:46.573693 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.573428    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" podStartSLOduration=6.573407092 podStartE2EDuration="6.573407092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:46.54725078 +0000 UTC m=+7.578987895" watchObservedRunningTime="2026-10-31 00:32:46.573407092 +0000 UTC m=+7.605144205"
Oct 31 00:32:46.601530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.601472    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hrtzv" podStartSLOduration=3.0556357690000002 podStartE2EDuration="7.601453875s" totalImagesPullingTime="4.545818106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:46.587310627 +0000 UTC m=+7.619047740" watchObservedRunningTime="2026-10-31 00:32:46.601453875 +0000 UTC m=+7.633190991"
Oct 31 00:32:46.883305 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.883206    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:46.883778 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.883654    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:46.902383 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:46.901205    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal" podStartSLOduration=6.901188905 podStartE2EDuration="6.901188905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:46.602161205 +0000 UTC m=+7.633898321" watchObservedRunningTime="2026-10-31 00:32:46.901188905 +0000 UTC m=+7.932926017"
Oct 31 00:32:47.551274 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:47.551192    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6hznk" event={"ID":"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff","Type":"ContainerStarted","Data":"2810c952e6cd3be334bd5355e30325c935147dd726e69c86f3c6ca79e527697f"}
Oct 31 00:32:48.118095 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.118063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:48.118253 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.118113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:48.118326 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.118254    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:48.118326 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.118303    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret podName:4c562d34-6b68-4800-a5a6-5c9e204a6050 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:56.118291164 +0000 UTC m=+17.150028271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret") pod "global-pull-secret-syncer-6gbrr" (UID: "4c562d34-6b68-4800-a5a6-5c9e204a6050") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:48.118669 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.118638    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:48.118788 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.118682    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:32:56.118672191 +0000 UTC m=+17.150409283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:48.219572 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.219488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:48.219725 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.219656    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:32:48.219725 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.219677    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:32:48.219725 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.219691    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xglkq for pod openshift-network-diagnostics/network-check-target-kwth8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:48.219897 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.219767    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq podName:007129ea-79e7-4781-a23c-8491d433c8a9 nodeName:}" failed. No retries permitted until 2026-10-31 00:32:56.219732721 +0000 UTC m=+17.251469828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xglkq" (UniqueName: "kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq") pod "network-check-target-kwth8" (UID: "007129ea-79e7-4781-a23c-8491d433c8a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:48.421530 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.421499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:48.421762 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.421617    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:48.422020 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.421998    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:48.422147 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.422117    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:48.422214 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.422184    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:48.422274 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:48.422254    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:48.556010 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.555973    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:32:48.556523 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:48.556440    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp" event={"ID":"bb393179-7795-4aa8-bd36-538fe2821808","Type":"ContainerStarted","Data":"81dafce9cacfe8ed1f0395bde2286e00e953c99e5f40bc3dc3d086df2b3f1210"}
Oct 31 00:32:49.357228 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.357195    2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 00:32:49.545696 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.545563    2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T00:32:49.357225063Z","UUID":"a0692cf8-e2c1-4e4c-98f6-efa2da1e2549","Handler":null,"Name":"","Endpoint":""}
Oct 31 00:32:49.550943 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.550919    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 00:32:49.551087 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.550950    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 00:32:49.563681 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.563615    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f4ddz" event={"ID":"fce6d408-267e-49f1-ac1e-68176943fbf9","Type":"ContainerStarted","Data":"9c37b6bb485483897e61d95bdf4543fb62e925b62b829ef0a602f1542f9e7531"}
Oct 31 00:32:49.567569 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.567541    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp" event={"ID":"bb393179-7795-4aa8-bd36-538fe2821808","Type":"ContainerStarted","Data":"917448af17c87ba304c245135f7cd8a44629393035a4f7613f0e8f1563cd6631"}
Oct 31 00:32:49.583300 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:49.583265    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6hznk" podStartSLOduration=3.947791231 podStartE2EDuration="9.583207777s" totalImagesPullingTime="5.635416546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:47.586733563 +0000 UTC m=+8.618470679" watchObservedRunningTime="2026-10-31 00:32:49.583207777 +0000 UTC m=+10.614944889"
Oct 31 00:32:50.420858 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.420829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:50.420996 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:50.420944    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:50.421058 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.421010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:50.421108 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:50.421080    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:50.421167 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.421134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:50.421260 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:50.421243    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:50.572208 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.572175    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d66sh" event={"ID":"edb0ea49-807e-4139-a8bc-516f3b88b720","Type":"ContainerStarted","Data":"f4db677ed818dd73a0baec4c2a1d1ab94a3458f2330f25bf68c3619e597c1a01"}
Oct 31 00:32:50.575285 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.575255    2578 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="c630a380db9891c410d7511570a1f6f864db18b147d663121f762addbd218a45" exitCode=0
Oct 31 00:32:50.575571 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.575549    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerDied","Data":"c630a380db9891c410d7511570a1f6f864db18b147d663121f762addbd218a45"}
Oct 31 00:32:50.580232 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.580204    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbgzq" event={"ID":"15983cd6-0227-4191-8770-4fd87b6a52b2","Type":"ContainerStarted","Data":"ca88e9e676a91936dd061648b248ea27a97c3aa84f03a0c484433071fc7a6dc1"}
Oct 31 00:32:50.590939 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.590904    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d66sh" podStartSLOduration=3.354823408 podStartE2EDuration="11.5908925s" totalImagesPullingTime="8.236069092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:50.590801479 +0000 UTC m=+11.622538592" watchObservedRunningTime="2026-10-31 00:32:50.5908925 +0000 UTC m=+11.622629614"
Oct 31 00:32:50.591335 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:50.591313    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-f4ddz" podStartSLOduration=3.744337558 podStartE2EDuration="11.591304875s" totalImagesPullingTime="7.846967317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:49.583619735 +0000 UTC m=+10.615356847" watchObservedRunningTime="2026-10-31 00:32:50.591304875 +0000 UTC m=+11.623041989"
Oct 31 00:32:51.586799 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.586505    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp" event={"ID":"bb393179-7795-4aa8-bd36-538fe2821808","Type":"ContainerStarted","Data":"65f20024150ba07e4aff88cf1c2a0f4071b0a0b7d3e72807d1a6cd37d425543b"}
Oct 31 00:32:51.588061 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.588032    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x2sfq" event={"ID":"dfde58a0-def5-41fb-8f2d-e825a8ab34fe","Type":"ContainerStarted","Data":"f81fe593eb0b9eb582c53b068518f1edaeee088d314cc45d7f7791a88cf92f79"}
Oct 31 00:32:51.590010 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.589986    2578 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="16200c0c728bcb654c2baaffb6ed245d450e004695889ea809ada4f43300486a" exitCode=0
Oct 31 00:32:51.590101 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.590065    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerDied","Data":"16200c0c728bcb654c2baaffb6ed245d450e004695889ea809ada4f43300486a"}
Oct 31 00:32:51.606123 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.606078    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp" podStartSLOduration=3.570306274 podStartE2EDuration="12.606067005s" totalImagesPullingTime="9.035760731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:51.606063589 +0000 UTC m=+12.637800736" watchObservedRunningTime="2026-10-31 00:32:51.606067005 +0000 UTC m=+12.637804118"
Oct 31 00:32:51.606265 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.606247    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-gbgzq" podStartSLOduration=4.394558394 podStartE2EDuration="12.606240075s" totalImagesPullingTime="8.211681681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:50.633794657 +0000 UTC m=+11.665531771" watchObservedRunningTime="2026-10-31 00:32:51.606240075 +0000 UTC m=+12.637977188"
Oct 31 00:32:51.622362 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:51.622324    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x2sfq" podStartSLOduration=2.302563986 podStartE2EDuration="12.622317215s" totalImagesPullingTime="10.319753229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:51.621835887 +0000 UTC m=+12.653573001" watchObservedRunningTime="2026-10-31 00:32:51.622317215 +0000 UTC m=+12.654054327"
Oct 31 00:32:52.420831 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:52.420785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:52.420831 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:52.420820    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:52.420997 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:52.420903    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:52.421102 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:52.421080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:52.421249 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:52.421198    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:52.421320 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:52.421290    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:53.597055 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:53.597015    2578 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="ac402c0a5472325d9f8707e8dfc41f88579dc8108a975424547fe1a3b89064e4" exitCode=0
Oct 31 00:32:53.597818 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:53.597074    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerDied","Data":"ac402c0a5472325d9f8707e8dfc41f88579dc8108a975424547fe1a3b89064e4"}
Oct 31 00:32:54.421607 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:54.421563    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:54.421607 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:54.421599    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:54.421872 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:54.421719    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:54.421872 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:54.421852    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:54.421943 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:54.421926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:54.422017 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:54.421999    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:54.788269 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:54.788230    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:54.788771 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:54.788378    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:32:54.789329 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:54.789308    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:32:56.184265 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.184061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:56.184265 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.184273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:56.184807 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.184200    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:56.184807 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.184379    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:56.184807 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.184384    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.184367904 +0000 UTC m=+33.216105010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:32:56.184807 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.184418    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret podName:4c562d34-6b68-4800-a5a6-5c9e204a6050 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.184408587 +0000 UTC m=+33.216145680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret") pod "global-pull-secret-syncer-6gbrr" (UID: "4c562d34-6b68-4800-a5a6-5c9e204a6050") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:32:56.285292 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.285247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:56.285438 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.285411    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:32:56.285438 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.285432    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:32:56.285525 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.285443    2578 projected.go:196] Error preparing data for projected volume kube-api-access-xglkq for pod openshift-network-diagnostics/network-check-target-kwth8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:56.285525 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.285493    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq podName:007129ea-79e7-4781-a23c-8491d433c8a9 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.285480614 +0000 UTC m=+33.317217710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xglkq" (UniqueName: "kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq") pod "network-check-target-kwth8" (UID: "007129ea-79e7-4781-a23c-8491d433c8a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:32:56.421423 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.421398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:56.421517 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.421469    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:56.421562 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.421551    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:56.421601 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.421567    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:56.421639 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.421628    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:56.421703 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:56.421687    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:56.606202 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606173    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:32:56.606553 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606530    2578 generic.go:356] "Generic (PLEG): container finished" podID="6643e3d6-52ac-4e9d-af24-8a18f92b3748" containerID="001be0ff76a940006bcb72c4e4d8842751ab9bde35c7086efe176e0ba583a514" exitCode=1
Oct 31 00:32:56.606653 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606599    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"a75b39f0fa1ba65d67b7e178d9ff0b0d56b4d0031e1dd196ee33932f8ddb9150"}
Oct 31 00:32:56.606718 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606667    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"acdb74aca99bdaae5e7b3f20db7986da8a8ad204802c3b5fdec2cba8c9470c6f"}
Oct 31 00:32:56.606718 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606710    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"532d6c540cb0e50db8b1d8290b4cdee39505e86c57b94f0962f6353a5c6a46bb"}
Oct 31 00:32:56.606824 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606777    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"d2f228ef15837be869c8d2d8c2f07a827b0dd4be8730ba88052738c6cbcdaa8c"}
Oct 31 00:32:56.606824 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606811    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerDied","Data":"001be0ff76a940006bcb72c4e4d8842751ab9bde35c7086efe176e0ba583a514"}
Oct 31 00:32:56.606911 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.606849    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"d0db626bbfd3f08411f4957606e0331ce2fa32785f3cb6f1c04d1159cc9c4b42"}
Oct 31 00:32:56.609538 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.609518    2578 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="69574eb4782ce89b3d4ad81c90701d2dbe38b83d562bfe2f1139df7e6768f30d" exitCode=0
Oct 31 00:32:56.609655 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:56.609548    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerDied","Data":"69574eb4782ce89b3d4ad81c90701d2dbe38b83d562bfe2f1139df7e6768f30d"}
Oct 31 00:32:57.614451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:57.614261    2578 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="c648669e59eaf2d143b3740aca8993af1ac6d588b136b7b7a67ab3e470d6617c" exitCode=0
Oct 31 00:32:57.614451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:57.614317    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerDied","Data":"c648669e59eaf2d143b3740aca8993af1ac6d588b136b7b7a67ab3e470d6617c"}
Oct 31 00:32:58.421423 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:58.421395    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:32:58.421423 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:58.421422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:32:58.421681 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:58.421395    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:32:58.421681 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:58.421513    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:32:58.421681 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:58.421627    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:32:58.421879 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:32:58.421685    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:32:58.620679 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:58.620640    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" event={"ID":"9557d837-f593-4d62-ad04-07b2c0b77987","Type":"ContainerStarted","Data":"f0105aa9afcd4c940a19fd0fc8ceba3ec16f86bb5a10e775a83a1c6353c661d0"}
Oct 31 00:32:58.646581 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:58.646529    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9tdv7" podStartSLOduration=8.755857601 podStartE2EDuration="19.646495691s" totalImagesPullingTime="10.89063809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:58.644974017 +0000 UTC m=+19.676711131" watchObservedRunningTime="2026-10-31 00:32:58.646495691 +0000 UTC m=+19.678232803"
Oct 31 00:32:59.625349 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:59.625321    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:32:59.625835 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:32:59.625627    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"6d20e949845a3a9914ee4a5f0fda06f29df40403f2fdf19c101d4a38c4334ade"}
Oct 31 00:33:00.421101 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:00.421057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:00.421250 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:00.421064    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:00.421250 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:00.421193    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:33:00.421309 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:00.421072    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:00.421309 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:00.421265    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:33:00.421390 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:00.421334    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:33:01.633186 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:01.633162    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:33:01.633941 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:01.633492    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"97fe0099b7e7b3b96e8d4c57544651a4b7dc2ea0f7651c5f444a9448959d4f4f"}
Oct 31 00:33:01.633941 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:01.633775    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:01.633941 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:01.633844    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:01.634113 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:01.633983    2578 scope.go:122] "RemoveContainer" containerID="001be0ff76a940006bcb72c4e4d8842751ab9bde35c7086efe176e0ba583a514"
Oct 31 00:33:01.650604 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:01.650579    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:02.421028 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.420995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:02.421028 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.421027    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:02.421215 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:02.421112    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:33:02.421215 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.421142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:02.421322 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:02.421212    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:33:02.421378 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:02.421332    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:33:02.465836 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.465803    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6gbrr"]
Oct 31 00:33:02.469294 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.469262    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5659t"]
Oct 31 00:33:02.469883 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.469860    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kwth8"]
Oct 31 00:33:02.639759 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.639718    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:33:02.640171 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.640143    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:02.640231 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.640194    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" event={"ID":"6643e3d6-52ac-4e9d-af24-8a18f92b3748","Type":"ContainerStarted","Data":"9bb8f7d5e68e93794bf0909d5470aa5d5a1fc70826a17054c2639641eda2b21f"}
Oct 31 00:33:02.640263 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:02.640240    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:33:02.640306 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.640260    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:02.640306 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.640297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:02.640403 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:02.640374    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:33:02.640547 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:02.640474    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:33:02.640658 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.640642    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:02.656114 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.656087    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:02.670930 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:02.670881    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk" podStartSLOduration=8.293897095 podStartE2EDuration="23.670866738s" totalImagesPullingTime="15.376969643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:02.668961711 +0000 UTC m=+23.700698825" watchObservedRunningTime="2026-10-31 00:33:02.670866738 +0000 UTC m=+23.702603852"
Oct 31 00:33:04.421552 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:04.421519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:04.422051 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:04.421606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:04.422051 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:04.421686    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:04.422051 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:04.421691    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:33:04.422051 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:04.421761    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:33:04.422051 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:04.421843    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:33:06.421676 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:06.421635    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:06.422160 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:06.421649    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:06.422160 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:06.421766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:06.422160 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:06.421761    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:33:06.422160 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:06.421837    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:33:06.422160 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:06.421907    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:33:08.420828 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.420793    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:08.421357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.420794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:08.421357 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:08.420929    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6gbrr" podUID="4c562d34-6b68-4800-a5a6-5c9e204a6050"
Oct 31 00:33:08.421357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.420965    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:08.421357 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:08.421046    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kwth8" podUID="007129ea-79e7-4781-a23c-8491d433c8a9"
Oct 31 00:33:08.421357 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:08.421112    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5659t" podUID="03f1dde4-3764-4c15-821f-54ca5cd3b47f"
Oct 31 00:33:08.774536 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.774509    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeReady"
Oct 31 00:33:08.774681 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.774646    2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 00:33:08.809526 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.809499    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-llgrf"]
Oct 31 00:33:08.812937 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.812908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:08.817159 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.817132    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"]
Oct 31 00:33:08.822640 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.822603    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.822801 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.822687    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.822867 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.822803    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8dbtk"]
Oct 31 00:33:08.823903 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.823018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:08.825899 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.825880    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64d4b7bcd7-2dvsg"]
Oct 31 00:33:08.826047 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.826031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:08.829223 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.829202    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-55fsb"]
Oct 31 00:33:08.829345 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.829329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:08.832737 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.832717    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ff66f9888-h24kh"]
Oct 31 00:33:08.833439 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.832916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:08.835642 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.835622    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"]
Oct 31 00:33:08.835777 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.835759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.838385 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.838368    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 31 00:33:08.838477 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.838463    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-85kwq\""
Oct 31 00:33:08.838681 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.838665    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"]
Oct 31 00:33:08.838841 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.838824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:08.840326 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.840306    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 31 00:33:08.840602 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.840581    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 31 00:33:08.841156 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.841126    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 31 00:33:08.841272 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.841176    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nzws7\""
Oct 31 00:33:08.841446 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.841427    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"]
Oct 31 00:33:08.842770 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.841732    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:08.848780 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.848736    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"]
Oct 31 00:33:08.848901 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.848840    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:08.851176 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.851053    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"]
Oct 31 00:33:08.851473 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.851454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:08.858475 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.858433    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"]
Oct 31 00:33:08.860274 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.859954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:08.867327 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.867303    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f887cdddf-kb24z"]
Oct 31 00:33:08.868476 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.868454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:08.873336 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.873267    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-gk844"]
Oct 31 00:33:08.873403 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.873393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.876038 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.876009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4smf\" (UniqueName: \"kubernetes.io/projected/ab123256-37b2-4cf7-961b-38c7a0e3be21-kube-api-access-b4smf\") pod \"kas-connection-checker-685895c7c9-llgrf\" (UID: \"ab123256-37b2-4cf7-961b-38c7a0e3be21\") " pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:08.876609 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.876592    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"]
Oct 31 00:33:08.876729 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.876711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.879379 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879360    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-llgrf"]
Oct 31 00:33:08.879482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879393    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8dbtk"]
Oct 31 00:33:08.879482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879416    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"]
Oct 31 00:33:08.879482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879436    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-55fsb"]
Oct 31 00:33:08.879482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879459    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"]
Oct 31 00:33:08.879482 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879480    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"]
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879502    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-gk844"]
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879523    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ff66f9888-h24kh"]
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879558    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"]
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879580    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"]
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879601    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d4b7bcd7-2dvsg"]
Oct 31 00:33:08.879700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.879625    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-22xfm"]
Oct 31 00:33:08.883472 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.883455    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zmn6t"]
Oct 31 00:33:08.883593 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.883578    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:08.886359 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886344    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f887cdddf-kb24z"]
Oct 31 00:33:08.886452 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886369    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"]
Oct 31 00:33:08.886452 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886383    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"]
Oct 31 00:33:08.886452 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886398    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"]
Oct 31 00:33:08.886452 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886413    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22xfm"]
Oct 31 00:33:08.886452 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886427    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zmn6t"]
Oct 31 00:33:08.886646 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.886488    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:08.923294 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923270    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.923294 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923270    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 31 00:33:08.923581 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923560    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.923697 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923586    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 31 00:33:08.923697 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923633    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 31 00:33:08.923822 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923718    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.923822 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923730    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.923822 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923789    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.923967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923832    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 00:33:08.923967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923860    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.923967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923861    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.923967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923882    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 31 00:33:08.923967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923938    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.923967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.923968    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.924947 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.924932    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.925013 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.924963    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 31 00:33:08.925013 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.924980    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925011    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925018    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925029    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925038    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925041    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925069    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 00:33:08.925100 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925094    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 00:33:08.925315 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925115    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.925531 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925506    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.925609 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.925561    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.927193 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.927128    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:08.935566 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.935548    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 31 00:33:08.940605 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940588    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 31 00:33:08.940738 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940611    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 00:33:08.940738 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940640    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 31 00:33:08.940931 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940775    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 00:33:08.940931 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940800    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 31 00:33:08.940931 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940830    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 31 00:33:08.940931 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.940836    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 31 00:33:08.941141 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941096    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfl7t\""
Oct 31 00:33:08.941229 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941211    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 31 00:33:08.941283 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941226    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-x5t4j\""
Oct 31 00:33:08.941283 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941223    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 00:33:08.941283 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941258    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wmhfw\""
Oct 31 00:33:08.941423 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941337    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5xbxm\""
Oct 31 00:33:08.941423 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941396    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 00:33:08.941517 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941424    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t9skn\""
Oct 31 00:33:08.941517 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941440    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wm824\""
Oct 31 00:33:08.941601 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941516    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-z5p2q\""
Oct 31 00:33:08.941601 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.941529    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 00:33:08.942278 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942260    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 00:33:08.942383 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942267    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 31 00:33:08.942383 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942297    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-bw4k9\""
Oct 31 00:33:08.942498 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942326    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 00:33:08.942498 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942332    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kh74f\""
Oct 31 00:33:08.942498 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942424    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-hwf9w\""
Oct 31 00:33:08.942498 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.942473    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gfdx5\""
Oct 31 00:33:08.977284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:08.977284 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-default-certificate\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:08.977487 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef90678c-925d-49bb-acd6-0d2861437916-tmp-dir\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:08.977487 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977325    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.977487 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-snapshots\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.977487 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977448    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcpnr\" (UniqueName: \"kubernetes.io/projected/1dabbb8b-9e70-4c1c-92d7-04d112150395-kube-api-access-qcpnr\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:08.977487 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.977487 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqmwt\" (UniqueName: \"kubernetes.io/projected/995d046f-21e7-4152-bdf2-d0954c8ff37f-kube-api-access-pqmwt\") pod \"kas-connection-checker-685895c7c9-55fsb\" (UID: \"995d046f-21e7-4152-bdf2-d0954c8ff37f\") " pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:08.977700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c58bcb9-e087-48f9-91a7-cecacedd8242-serving-cert\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:08.977700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss65p\" (UniqueName: \"kubernetes.io/projected/3570c076-c980-454d-9b4d-4fbcc88953e0-kube-api-access-ss65p\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:08.977700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.977700 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977638    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:08.977953 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977697    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.977953 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:08.977953 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.977953 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.977953 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.977953 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4e15376-708a-442b-9439-8bec92eae34c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-stats-auth\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.977994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978048    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c58bcb9-e087-48f9-91a7-cecacedd8242-config\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.978212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmdf6\" (UniqueName: \"kubernetes.io/projected/7c58bcb9-e087-48f9-91a7-cecacedd8242-kube-api-access-rmdf6\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-config\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978242    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-trusted-ca\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978265    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dabbb8b-9e70-4c1c-92d7-04d112150395-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-serving-cert\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f593360-052d-45fe-8838-7395a5d16b21-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f593360-052d-45fe-8838-7395a5d16b21-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.978458 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-cert\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5h6s\" (UniqueName: \"kubernetes.io/projected/44d79171-f637-4834-bfaa-bd57932eb5b4-kube-api-access-j5h6s\") pod \"volume-data-source-validator-58c9f5f46d-9cltd\" (UID: \"44d79171-f637-4834-bfaa-bd57932eb5b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef90678c-925d-49bb-acd6-0d2861437916-config-volume\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t8cq\" (UniqueName: \"kubernetes.io/projected/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-kube-api-access-4t8cq\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4smf\" (UniqueName: \"kubernetes.io/projected/ab123256-37b2-4cf7-961b-38c7a0e3be21-kube-api-access-b4smf\") pod \"kas-connection-checker-685895c7c9-llgrf\" (UID: \"ab123256-37b2-4cf7-961b-38c7a0e3be21\") " pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:08.978834 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-tmp\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/041328ee-8f09-4927-85e3-f3cceefe3a53-serving-cert\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27m64\" (UniqueName: \"kubernetes.io/projected/2f593360-052d-45fe-8838-7395a5d16b21-kube-api-access-27m64\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zhwq\" (UniqueName: \"kubernetes.io/projected/65327963-1791-42b7-a9a7-4e77b700ed5a-kube-api-access-5zhwq\") pod \"kas-connection-checker-685895c7c9-8dbtk\" (UID: \"65327963-1791-42b7-a9a7-4e77b700ed5a\") " pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdptw\" (UniqueName: \"kubernetes.io/projected/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-kube-api-access-kdptw\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.978975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-service-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.979008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg77m\" (UniqueName: \"kubernetes.io/projected/ef90678c-925d-49bb-acd6-0d2861437916-kube-api-access-sg77m\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.979032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwdwd\" (UniqueName: \"kubernetes.io/projected/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-kube-api-access-kwdwd\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.979064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb9b9\" (UniqueName: \"kubernetes.io/projected/041328ee-8f09-4927-85e3-f3cceefe3a53-kube-api-access-bb9b9\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.979110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:08.979286 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.979138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkv5h\" (UniqueName: \"kubernetes.io/projected/38dad442-b12d-4dc8-9f95-1471ff276929-kube-api-access-fkv5h\") pod \"network-check-source-6f78d59f99-j5zl7\" (UID: \"38dad442-b12d-4dc8-9f95-1471ff276929\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:08.995384 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:08.995357    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4smf\" (UniqueName: \"kubernetes.io/projected/ab123256-37b2-4cf7-961b-38c7a0e3be21-kube-api-access-b4smf\") pod \"kas-connection-checker-685895c7c9-llgrf\" (UID: \"ab123256-37b2-4cf7-961b-38c7a0e3be21\") " pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:09.080005 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.079930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-serving-cert\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.080005 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.079961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080005 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.079979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f593360-052d-45fe-8838-7395a5d16b21-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.080005 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080001    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f593360-052d-45fe-8838-7395a5d16b21-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-cert\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080268    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5h6s\" (UniqueName: \"kubernetes.io/projected/44d79171-f637-4834-bfaa-bd57932eb5b4-kube-api-access-j5h6s\") pod \"volume-data-source-validator-58c9f5f46d-9cltd\" (UID: \"44d79171-f637-4834-bfaa-bd57932eb5b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:09.080321 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef90678c-925d-49bb-acd6-0d2861437916-config-volume\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t8cq\" (UniqueName: \"kubernetes.io/projected/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-kube-api-access-4t8cq\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-tmp\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/041328ee-8f09-4927-85e3-f3cceefe3a53-serving-cert\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27m64\" (UniqueName: \"kubernetes.io/projected/2f593360-052d-45fe-8838-7395a5d16b21-kube-api-access-27m64\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080542    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zhwq\" (UniqueName: \"kubernetes.io/projected/65327963-1791-42b7-a9a7-4e77b700ed5a-kube-api-access-5zhwq\") pod \"kas-connection-checker-685895c7c9-8dbtk\" (UID: \"65327963-1791-42b7-a9a7-4e77b700ed5a\") " pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdptw\" (UniqueName: \"kubernetes.io/projected/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-kube-api-access-kdptw\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:09.080648 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-service-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg77m\" (UniqueName: \"kubernetes.io/projected/ef90678c-925d-49bb-acd6-0d2861437916-kube-api-access-sg77m\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwdwd\" (UniqueName: \"kubernetes.io/projected/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-kube-api-access-kwdwd\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9b9\" (UniqueName: \"kubernetes.io/projected/041328ee-8f09-4927-85e3-f3cceefe3a53-kube-api-access-bb9b9\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.080730    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080731    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f593360-052d-45fe-8838-7395a5d16b21-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.080760    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkv5h\" (UniqueName: \"kubernetes.io/projected/38dad442-b12d-4dc8-9f95-1471ff276929-kube-api-access-fkv5h\") pod \"network-check-source-6f78d59f99-j5zl7\" (UID: \"38dad442-b12d-4dc8-9f95-1471ff276929\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.080812    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.580793394 +0000 UTC m=+30.612530503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-default-certificate\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef90678c-925d-49bb-acd6-0d2861437916-tmp-dir\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-snapshots\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.080985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpnr\" (UniqueName: \"kubernetes.io/projected/1dabbb8b-9e70-4c1c-92d7-04d112150395-kube-api-access-qcpnr\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:09.081192 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmwt\" (UniqueName: \"kubernetes.io/projected/995d046f-21e7-4152-bdf2-d0954c8ff37f-kube-api-access-pqmwt\") pod \"kas-connection-checker-685895c7c9-55fsb\" (UID: \"995d046f-21e7-4152-bdf2-d0954c8ff37f\") " pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c58bcb9-e087-48f9-91a7-cecacedd8242-serving-cert\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss65p\" (UniqueName: \"kubernetes.io/projected/3570c076-c980-454d-9b4d-4fbcc88953e0-kube-api-access-ss65p\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081327    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-service-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4e15376-708a-442b-9439-8bec92eae34c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-stats-auth\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.083576 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c58bcb9-e087-48f9-91a7-cecacedd8242-config\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081708    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmdf6\" (UniqueName: \"kubernetes.io/projected/7c58bcb9-e087-48f9-91a7-cecacedd8242-kube-api-access-rmdf6\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-config\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-trusted-ca\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dabbb8b-9e70-4c1c-92d7-04d112150395-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.081977    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.082075    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.082152    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.582134184 +0000 UTC m=+30.613871290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.082206    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.582185589 +0000 UTC m=+30.613922704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.082299    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.082368    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.582353435 +0000 UTC m=+30.614090527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:09.084451 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.082450    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-tmp\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.082497    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef90678c-925d-49bb-acd6-0d2861437916-config-volume\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.083516    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.083562    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.583548675 +0000 UTC m=+30.615285783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.083600    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f593360-052d-45fe-8838-7395a5d16b21-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.083830    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.084212    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-serving-cert\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.084296    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.084342    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.584328073 +0000 UTC m=+30.616065179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.084775    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dabbb8b-9e70-4c1c-92d7-04d112150395-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.085013    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.085370 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.085325    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-config\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.085966 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.085378    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.085966 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.085400    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4e15376-708a-442b-9439-8bec92eae34c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:09.085966 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.085409    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.085966 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.085490    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-cert\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:09.086226 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086133    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/041328ee-8f09-4927-85e3-f3cceefe3a53-serving-cert\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.086226 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086161    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.086226 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086164    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c58bcb9-e087-48f9-91a7-cecacedd8242-config\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.086583 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086236    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-trusted-ca\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.086583 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086256    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.086583 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086413    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.086583 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086509    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef90678c-925d-49bb-acd6-0d2861437916-tmp-dir\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.086583 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.086553    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:09.086860 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.086599    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.586585678 +0000 UTC m=+30.618322783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:09.086860 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.086722    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:09.086860 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.086739    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:09.086860 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086789    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-snapshots\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.086860 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.086822    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:09.087107 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.086956    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:09.586938099 +0000 UTC m=+30.618675208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:09.088798 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.088778    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-default-certificate\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.089423 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.089404    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c58bcb9-e087-48f9-91a7-cecacedd8242-serving-cert\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.089527 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.089482    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.090126 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.090098    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-stats-auth\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.090210 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.090131    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zhwq\" (UniqueName: \"kubernetes.io/projected/65327963-1791-42b7-a9a7-4e77b700ed5a-kube-api-access-5zhwq\") pod \"kas-connection-checker-685895c7c9-8dbtk\" (UID: \"65327963-1791-42b7-a9a7-4e77b700ed5a\") " pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:09.090342 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.090323    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5h6s\" (UniqueName: \"kubernetes.io/projected/44d79171-f637-4834-bfaa-bd57932eb5b4-kube-api-access-j5h6s\") pod \"volume-data-source-validator-58c9f5f46d-9cltd\" (UID: \"44d79171-f637-4834-bfaa-bd57932eb5b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:09.091863 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.091832    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkv5h\" (UniqueName: \"kubernetes.io/projected/38dad442-b12d-4dc8-9f95-1471ff276929-kube-api-access-fkv5h\") pod \"network-check-source-6f78d59f99-j5zl7\" (UID: \"38dad442-b12d-4dc8-9f95-1471ff276929\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:09.093883 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.093814    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.094255 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.094232    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss65p\" (UniqueName: \"kubernetes.io/projected/3570c076-c980-454d-9b4d-4fbcc88953e0-kube-api-access-ss65p\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.094414 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.094391    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwdwd\" (UniqueName: \"kubernetes.io/projected/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-kube-api-access-kwdwd\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.094566 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.094548    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t8cq\" (UniqueName: \"kubernetes.io/projected/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-kube-api-access-4t8cq\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:09.095444 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.095421    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmdf6\" (UniqueName: \"kubernetes.io/projected/7c58bcb9-e087-48f9-91a7-cecacedd8242-kube-api-access-rmdf6\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.095902 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.095882    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdptw\" (UniqueName: \"kubernetes.io/projected/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-kube-api-access-kdptw\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:09.099278 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.099255    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9b9\" (UniqueName: \"kubernetes.io/projected/041328ee-8f09-4927-85e3-f3cceefe3a53-kube-api-access-bb9b9\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.100952 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.100930    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27m64\" (UniqueName: \"kubernetes.io/projected/2f593360-052d-45fe-8838-7395a5d16b21-kube-api-access-27m64\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.101026 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.100959    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.101195 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.101175    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.101441 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.101423    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.108820 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.108801    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmwt\" (UniqueName: \"kubernetes.io/projected/995d046f-21e7-4152-bdf2-d0954c8ff37f-kube-api-access-pqmwt\") pod \"kas-connection-checker-685895c7c9-55fsb\" (UID: \"995d046f-21e7-4152-bdf2-d0954c8ff37f\") " pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:09.109212 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.109195    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg77m\" (UniqueName: \"kubernetes.io/projected/ef90678c-925d-49bb-acd6-0d2861437916-kube-api-access-sg77m\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.109277 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.109256    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpnr\" (UniqueName: \"kubernetes.io/projected/1dabbb8b-9e70-4c1c-92d7-04d112150395-kube-api-access-qcpnr\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:09.128541 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.128507    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:09.140625 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.140602    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:09.162157 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.162127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:09.178459 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.178429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:09.191921 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.191894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:09.208025 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.203616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:09.208025 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.204255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:09.222115 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.222022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:09.246126 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.246095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:09.254196 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.253987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:09.428221 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.428188    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"]
Oct 31 00:33:09.432820 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.432792    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f593360_052d_45fe_8838_7395a5d16b21.slice/crio-7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed WatchSource:0}: Error finding container 7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed: Status 404 returned error can't find the container with id 7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed
Oct 31 00:33:09.437878 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.437855    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab123256_37b2_4cf7_961b_38c7a0e3be21.slice/crio-e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7 WatchSource:0}: Error finding container e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7: Status 404 returned error can't find the container with id e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7
Oct 31 00:33:09.444390 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.440803    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-llgrf"]
Oct 31 00:33:09.465669 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.465635    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8dbtk"]
Oct 31 00:33:09.468106 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.468077    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65327963_1791_42b7_a9a7_4e77b700ed5a.slice/crio-4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891 WatchSource:0}: Error finding container 4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891: Status 404 returned error can't find the container with id 4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891
Oct 31 00:33:09.497357 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.497328    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-55fsb"]
Oct 31 00:33:09.512976 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.512945    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-gk844"]
Oct 31 00:33:09.521601 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.521548    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9c0bff_850c_4b57_b3fe_ea825f6d4767.slice/crio-5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d WatchSource:0}: Error finding container 5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d: Status 404 returned error can't find the container with id 5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d
Oct 31 00:33:09.553088 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.553064    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"]
Oct 31 00:33:09.555943 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.555905    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041328ee_8f09_4927_85e3_f3cceefe3a53.slice/crio-1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28 WatchSource:0}: Error finding container 1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28: Status 404 returned error can't find the container with id 1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28
Oct 31 00:33:09.562436 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.562402    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"]
Oct 31 00:33:09.564840 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.564811    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c58bcb9_e087_48f9_91a7_cecacedd8242.slice/crio-9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1 WatchSource:0}: Error finding container 9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1: Status 404 returned error can't find the container with id 9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1
Oct 31 00:33:09.576155 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.576128    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"]
Oct 31 00:33:09.578005 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.577980    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d79171_f637_4834_bfaa_bd57932eb5b4.slice/crio-32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0 WatchSource:0}: Error finding container 32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0: Status 404 returned error can't find the container with id 32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0
Oct 31 00:33:09.590233 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:09.590323 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:09.590371 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:09.590413 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:09.590413 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590383    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:09.590413 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590402    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590445    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590455    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590454    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590434243 +0000 UTC m=+31.622171356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590498    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590512    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590499196 +0000 UTC m=+31.622236288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:09.590526 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590526    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590520256 +0000 UTC m=+31.622257348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590542    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590533633 +0000 UTC m=+31.622270725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590563    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590623    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590606537 +0000 UTC m=+31.622343644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590658    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590669    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590696    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.5906894 +0000 UTC m=+31.622426521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.590715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590767    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590755573 +0000 UTC m=+31.622492679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590830    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:09.590884 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:09.590881    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:10.590867019 +0000 UTC m=+31.622604133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:09.616578 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.616557    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"]
Oct 31 00:33:09.618106 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.618082    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22xfm"]
Oct 31 00:33:09.618568 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.618538    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dad442_b12d_4dc8_9f95_1471ff276929.slice/crio-0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2 WatchSource:0}: Error finding container 0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2: Status 404 returned error can't find the container with id 0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2
Oct 31 00:33:09.619180 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:09.619108    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db3b75c_d1fb_4b85_bc90_0c9bcbafd9ee.slice/crio-670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e WatchSource:0}: Error finding container 670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e: Status 404 returned error can't find the container with id 670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e
Oct 31 00:33:09.656585 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.656554    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-gk844" event={"ID":"ff9c0bff-850c-4b57-b3fe-ea825f6d4767","Type":"ContainerStarted","Data":"5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d"}
Oct 31 00:33:09.657902 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.657880    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x" event={"ID":"2f593360-052d-45fe-8838-7395a5d16b21","Type":"ContainerStarted","Data":"7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed"}
Oct 31 00:33:09.659167 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.659141    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-llgrf" event={"ID":"ab123256-37b2-4cf7-961b-38c7a0e3be21","Type":"ContainerStarted","Data":"116fb9b62a03349ecb879cc2855e18b4530d47bee8937d49016d2527f9a5c37e"}
Oct 31 00:33:09.659253 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.659188    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-llgrf" event={"ID":"ab123256-37b2-4cf7-961b-38c7a0e3be21","Type":"ContainerStarted","Data":"e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7"}
Oct 31 00:33:09.660727 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.660682    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7" event={"ID":"38dad442-b12d-4dc8-9f95-1471ff276929","Type":"ContainerStarted","Data":"0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2"}
Oct 31 00:33:09.661795 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.661771    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" event={"ID":"041328ee-8f09-4927-85e3-f3cceefe3a53","Type":"ContainerStarted","Data":"1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28"}
Oct 31 00:33:09.663520 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.663503    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8dbtk" event={"ID":"65327963-1791-42b7-a9a7-4e77b700ed5a","Type":"ContainerStarted","Data":"e1df648da89bafe0ac6f50b109d3373947a6fc0a64b37e61fe09ffd35cd5c318"}
Oct 31 00:33:09.663601 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.663534    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8dbtk" event={"ID":"65327963-1791-42b7-a9a7-4e77b700ed5a","Type":"ContainerStarted","Data":"4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891"}
Oct 31 00:33:09.664905 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.664887    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-55fsb" event={"ID":"995d046f-21e7-4152-bdf2-d0954c8ff37f","Type":"ContainerStarted","Data":"df6c1e3693d868ee10ceccaaaf860d2ebbc5a45483ec0fd3692ab1e68f900278"}
Oct 31 00:33:09.665961 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.665941    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd" event={"ID":"44d79171-f637-4834-bfaa-bd57932eb5b4","Type":"ContainerStarted","Data":"32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0"}
Oct 31 00:33:09.667088 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.667064    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg" event={"ID":"7c58bcb9-e087-48f9-91a7-cecacedd8242","Type":"ContainerStarted","Data":"9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1"}
Oct 31 00:33:09.668181 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.668164    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22xfm" event={"ID":"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee","Type":"ContainerStarted","Data":"670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e"}
Oct 31 00:33:09.677991 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.677950    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-llgrf" podStartSLOduration=384.677935529 podStartE2EDuration="6m24.677935529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:09.677284084 +0000 UTC m=+30.709021198" watchObservedRunningTime="2026-10-31 00:33:09.677935529 +0000 UTC m=+30.709672646"
Oct 31 00:33:09.694874 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:09.694835    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-8dbtk" podStartSLOduration=384.694823469 podStartE2EDuration="6m24.694823469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:09.693865787 +0000 UTC m=+30.725602897" watchObservedRunningTime="2026-10-31 00:33:09.694823469 +0000 UTC m=+30.726560576"
Oct 31 00:33:10.422531 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.421481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:10.422531 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.421845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:10.422531 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.422072    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:10.427670 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.426916    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-v94lw\""
Oct 31 00:33:10.427670 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.427052    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:33:10.427670 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.427138    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-qhb2k\""
Oct 31 00:33:10.427670 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.427218    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-87knn\""
Oct 31 00:33:10.427670 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.427271    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:33:10.600257 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:10.600675 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.600568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:10.600926 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.600708    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:10.600926 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.600800    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.60078133 +0000 UTC m=+33.632518436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:10.601229 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601211    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:10.601287 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601233    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:10.601287 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601279    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.601264879 +0000 UTC m=+33.633001985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:10.601355 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601338    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:10.601405 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601371    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.601360287 +0000 UTC m=+33.633097396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:10.601444 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601421    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:10.601479 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601452    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.601442599 +0000 UTC m=+33.633179706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:10.601514 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601503    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:10.601548 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601529    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.601520058 +0000 UTC m=+33.633257164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:10.601593 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601585    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:10.601630 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601614    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.601605278 +0000 UTC m=+33.633342384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:10.602145 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601671    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:10.602145 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601684    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:10.602145 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.601714    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.601704314 +0000 UTC m=+33.633441422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:10.602145 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:10.602102    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:12.60208805 +0000 UTC m=+33.633825165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:10.676160 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.676061    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-55fsb" event={"ID":"995d046f-21e7-4152-bdf2-d0954c8ff37f","Type":"ContainerStarted","Data":"18e5210d3fa27078ba44dcdd4c60fcbd9a6b680661a576b6f2fa6ce5c3a1c396"}
Oct 31 00:33:10.694694 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:10.694599    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-55fsb" podStartSLOduration=385.694584314 podStartE2EDuration="6m25.694584314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:10.693601568 +0000 UTC m=+31.725338708" watchObservedRunningTime="2026-10-31 00:33:10.694584314 +0000 UTC m=+31.726321486"
Oct 31 00:33:12.222026 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.221234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:12.222026 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.221317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:12.222026 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.221427    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 00:33:12.222026 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.221473    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:33:44.221461352 +0000 UTC m=+65.253198444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : secret "metrics-daemon-secret" not found
Oct 31 00:33:12.226427 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.226401    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:12.261169 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.261136    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:12.322069 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.322033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:12.326122 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.326016    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:12.600081 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.598979    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6gbrr"]
Oct 31 00:33:12.605960 ip-10-0-142-217 kubenswrapper[2578]: W1031 00:33:12.605249    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c562d34_6b68_4800_a5a6_5c9e204a6050.slice/crio-f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4 WatchSource:0}: Error finding container f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4: Status 404 returned error can't find the container with id f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4
Oct 31 00:33:12.608513 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.608436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:12.625492 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:12.625680 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:12.625680 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:12.625680 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:12.625680 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:12.625680 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:12.626052 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:12.626052 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.625768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:12.626052 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.625873    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:12.626052 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.625918    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.625905628 +0000 UTC m=+37.657642720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626308    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626366    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.626349943 +0000 UTC m=+37.658087035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626415    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626437    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.626431068 +0000 UTC m=+37.658168160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626475    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626495    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.626489311 +0000 UTC m=+37.658226403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626531    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626538    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626556    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.62655051 +0000 UTC m=+37.658287606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626594    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.626589242 +0000 UTC m=+37.658326335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626623    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626639    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.626634601 +0000 UTC m=+37.658371692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626687    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626696    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:12.626774 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:12.626723    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.626714476 +0000 UTC m=+37.658451571 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:12.687231 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.686469    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd" event={"ID":"44d79171-f637-4834-bfaa-bd57932eb5b4","Type":"ContainerStarted","Data":"11197290ca77beac2b22741852a1e25c0cd7eede30b39db719d82ade1eba5779"}
Oct 31 00:33:12.689075 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.689002    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6gbrr" event={"ID":"4c562d34-6b68-4800-a5a6-5c9e204a6050","Type":"ContainerStarted","Data":"f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4"}
Oct 31 00:33:12.704394 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.704310    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd" podStartSLOduration=333.042177824 podStartE2EDuration="5m35.704292835s" totalImagesPullingTime="2.662115011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:12.70378044 +0000 UTC m=+33.735517555" watchObservedRunningTime="2026-10-31 00:33:12.704292835 +0000 UTC m=+33.736029953"
Oct 31 00:33:12.921178 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:12.916707    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kwth8"]
Oct 31 00:33:13.707844 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:13.707770    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kwth8" event={"ID":"007129ea-79e7-4781-a23c-8491d433c8a9","Type":"ContainerStarted","Data":"6380445fd64913146f6ba600aee8e5eacf66c16176567efa266f7e325c0e1c7c"}
Oct 31 00:33:15.722812 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.721597    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/0.log"
Oct 31 00:33:15.722812 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.721651    2578 generic.go:356] "Generic (PLEG): container finished" podID="041328ee-8f09-4927-85e3-f3cceefe3a53" containerID="7e56bb06f2a8b18d74331c1e4ebe00436f8387222e046d17caaddd4f55939250" exitCode=255
Oct 31 00:33:15.722812 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.721774    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" event={"ID":"041328ee-8f09-4927-85e3-f3cceefe3a53","Type":"ContainerDied","Data":"7e56bb06f2a8b18d74331c1e4ebe00436f8387222e046d17caaddd4f55939250"}
Oct 31 00:33:15.722812 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.722119    2578 scope.go:122] "RemoveContainer" containerID="7e56bb06f2a8b18d74331c1e4ebe00436f8387222e046d17caaddd4f55939250"
Oct 31 00:33:15.732276 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.732246    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg" event={"ID":"7c58bcb9-e087-48f9-91a7-cecacedd8242","Type":"ContainerStarted","Data":"a1644eef1f75f6a9d19f9e08ed03654af928331122ef5a9e5feff7e67ee3e5f4"}
Oct 31 00:33:15.735903 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.735361    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22xfm" event={"ID":"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee","Type":"ContainerStarted","Data":"4319686c1ed8daa4c9762aa9550ff41a48a0053430b6daf1949da10a42da02f1"}
Oct 31 00:33:15.739657 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.739125    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-gk844" event={"ID":"ff9c0bff-850c-4b57-b3fe-ea825f6d4767","Type":"ContainerStarted","Data":"cdcc19f0958c4c462de6ef56540aa6fbf63e2bde07d8a068db16ba9d9550a20a"}
Oct 31 00:33:15.745107 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.744594    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x" event={"ID":"2f593360-052d-45fe-8838-7395a5d16b21","Type":"ContainerStarted","Data":"fc291e6a13a0eceb15870a2f414df93b9db56e41142413579a26049cc56b2330"}
Oct 31 00:33:15.764380 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.764324    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x" podStartSLOduration=334.307943031 podStartE2EDuration="5m39.764310316s" totalImagesPullingTime="5.456367285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:15.763390323 +0000 UTC m=+36.795127454" watchObservedRunningTime="2026-10-31 00:33:15.764310316 +0000 UTC m=+36.796047431"
Oct 31 00:33:15.782827 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.781941    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg" podStartSLOduration=333.486225325 podStartE2EDuration="5m38.781921888s" totalImagesPullingTime="5.295696563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:15.781082214 +0000 UTC m=+36.812819329" watchObservedRunningTime="2026-10-31 00:33:15.781921888 +0000 UTC m=+36.813659003"
Oct 31 00:33:15.831113 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.826063    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-22xfm" podStartSLOduration=2.011440595 podStartE2EDuration="7.826044451s" totalImagesPullingTime="5.814603856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:15.80247444 +0000 UTC m=+36.834211566" watchObservedRunningTime="2026-10-31 00:33:15.826044451 +0000 UTC m=+36.857781567"
Oct 31 00:33:15.931636 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.931555    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-gk844" podStartSLOduration=337.593058065 podStartE2EDuration="5m42.931532774s" totalImagesPullingTime="5.338474709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:15.828147325 +0000 UTC m=+36.859884440" watchObservedRunningTime="2026-10-31 00:33:15.931532774 +0000 UTC m=+36.963269892"
Oct 31 00:33:15.932408 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.932382    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bcmv5"]
Oct 31 00:33:15.950040 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.950006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:15.956005 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.955932    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bcmv5"]
Oct 31 00:33:15.966681 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.966654    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 00:33:15.966802 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.966723    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 00:33:15.966883 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.966868    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6p62k\""
Oct 31 00:33:15.994131 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:15.994102    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"]
Oct 31 00:33:16.012405 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.012375    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:16.012405 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.012394    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"]
Oct 31 00:33:16.058815 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.058764    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pqlnb\""
Oct 31 00:33:16.059446 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.059425    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 00:33:16.059580 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.059513    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:16.071233 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.071196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.071399 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.071251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-crio-socket\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.071399 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.071328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8mpf\" (UniqueName: \"kubernetes.io/projected/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-api-access-s8mpf\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.071399 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.071373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.071556 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.071451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-data-volume\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.172153 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-data-volume\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.172508 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172488    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-data-volume\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.172630 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.172678 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-crio-socket\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.172852 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jx9h\" (UniqueName: \"kubernetes.io/projected/69a61c32-b459-46fa-a720-80637d99c7ae-kube-api-access-8jx9h\") pod \"migrator-66df5d496b-whnqr\" (UID: \"69a61c32-b459-46fa-a720-80637d99c7ae\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:16.172929 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8mpf\" (UniqueName: \"kubernetes.io/projected/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-api-access-s8mpf\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.172967 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.172925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.173092 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.173077    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:16.173154 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.173133    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.673116732 +0000 UTC m=+37.704853838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:16.173373 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.173360    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 00:33:16.173414 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.173400    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:16.673387714 +0000 UTC m=+37.705124819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : configmap "kube-rbac-proxy" not found
Oct 31 00:33:16.173454 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.173447    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-crio-socket\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.182495 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.182462    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8mpf\" (UniqueName: \"kubernetes.io/projected/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-api-access-s8mpf\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.274130 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.274089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jx9h\" (UniqueName: \"kubernetes.io/projected/69a61c32-b459-46fa-a720-80637d99c7ae-kube-api-access-8jx9h\") pod \"migrator-66df5d496b-whnqr\" (UID: \"69a61c32-b459-46fa-a720-80637d99c7ae\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:16.285663 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.285634    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jx9h\" (UniqueName: \"kubernetes.io/projected/69a61c32-b459-46fa-a720-80637d99c7ae-kube-api-access-8jx9h\") pod \"migrator-66df5d496b-whnqr\" (UID: \"69a61c32-b459-46fa-a720-80637d99c7ae\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:16.360097 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.360052    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:16.655445 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.655079    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"]
Oct 31 00:33:16.678110 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.678244 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:16.678244 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:16.678367 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:16.678367 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:16.678367 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:16.678367 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:16.678644 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:16.678644 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:16.678644 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.678457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:16.678644 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.678588    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:16.678856 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.678649    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.678631749 +0000 UTC m=+45.710368861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:16.678919 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.678887    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:16.678968 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.678945    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:17.678927469 +0000 UTC m=+38.710664574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:16.679023 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.678990    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 00:33:16.679023 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679021    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:17.679011586 +0000 UTC m=+38.710748679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : configmap "kube-rbac-proxy" not found
Oct 31 00:33:16.679117 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679033    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:16.679117 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679069    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.679059039 +0000 UTC m=+45.710796160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:16.679117 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679083    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:16.679117 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679095    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:16.679117 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679108    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679122    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.6791137 +0000 UTC m=+45.710850797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679138    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.679129247 +0000 UTC m=+45.710866343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679158    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679165    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679183    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.679177198 +0000 UTC m=+45.710914335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679202    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679223    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.679217319 +0000 UTC m=+45.710954458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679239    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.6792308 +0000 UTC m=+45.710967891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679268    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:16.679349 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.679298    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.679288662 +0000 UTC m=+45.711025758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:16.750873 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.750804    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7" event={"ID":"38dad442-b12d-4dc8-9f95-1471ff276929","Type":"ContainerStarted","Data":"3c20bf621cd172fd52ce4f832acfe03e6a16fb523c0a1ab8d3407919bf252198"}
Oct 31 00:33:16.753333 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.753310    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/1.log"
Oct 31 00:33:16.753784 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.753768    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/0.log"
Oct 31 00:33:16.753906 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.753809    2578 generic.go:356] "Generic (PLEG): container finished" podID="041328ee-8f09-4927-85e3-f3cceefe3a53" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a" exitCode=255
Oct 31 00:33:16.753906 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.753886    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" event={"ID":"041328ee-8f09-4927-85e3-f3cceefe3a53","Type":"ContainerDied","Data":"e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a"}
Oct 31 00:33:16.754018 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.753928    2578 scope.go:122] "RemoveContainer" containerID="7e56bb06f2a8b18d74331c1e4ebe00436f8387222e046d17caaddd4f55939250"
Oct 31 00:33:16.754195 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.754180    2578 scope.go:122] "RemoveContainer" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a"
Oct 31 00:33:16.754408 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:16.754386    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-f6w7r_openshift-console-operator(041328ee-8f09-4927-85e3-f3cceefe3a53)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53"
Oct 31 00:33:16.756952 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.756917    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr" event={"ID":"69a61c32-b459-46fa-a720-80637d99c7ae","Type":"ContainerStarted","Data":"e84748c0c29c4be6338fb082c311df6f65e039aaeb3fed3c5f62225fe9fcb1f5"}
Oct 31 00:33:16.774952 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:16.774873    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7" podStartSLOduration=334.19634502 podStartE2EDuration="5m40.774858132s" totalImagesPullingTime="6.578513112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:16.773327362 +0000 UTC m=+37.805064505" watchObservedRunningTime="2026-10-31 00:33:16.774858132 +0000 UTC m=+37.806595245"
Oct 31 00:33:17.689096 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.689057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:17.689285 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.689184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:17.689285 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:17.689194    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 00:33:17.689285 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:17.689257    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:19.689239754 +0000 UTC m=+40.720976859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : configmap "kube-rbac-proxy" not found
Oct 31 00:33:17.689420 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:17.689331    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:17.689420 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:17.689370    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:19.689359219 +0000 UTC m=+40.721096326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:17.767490 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.767393    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kwth8" event={"ID":"007129ea-79e7-4781-a23c-8491d433c8a9","Type":"ContainerStarted","Data":"81238f41f82411fa7823c949721350e9089da38c53a22e01f660662c9f1a64cf"}
Oct 31 00:33:17.768014 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.767562    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:17.769765 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.769728    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/1.log"
Oct 31 00:33:17.770239 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.770220    2578 scope.go:122] "RemoveContainer" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a"
Oct 31 00:33:17.770428 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:17.770404    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-f6w7r_openshift-console-operator(041328ee-8f09-4927-85e3-f3cceefe3a53)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53"
Oct 31 00:33:17.786945 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:17.786889    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kwth8" podStartSLOduration=34.995899015 podStartE2EDuration="38.786875158s" totalImagesPullingTime="3.790976143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:17.78591612 +0000 UTC m=+38.817653233" watchObservedRunningTime="2026-10-31 00:33:17.786875158 +0000 UTC m=+38.818612273"
Oct 31 00:33:19.222779 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:19.222679    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:19.222779 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:19.222740    2578 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:19.223290 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:19.223182    2578 scope.go:122] "RemoveContainer" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a"
Oct 31 00:33:19.223422 ip-10-0-142-217 kubenswrapper[2578]: E1031 00:33:19.223400    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-f6w7r_openshift-console-operator(041328ee-8f09-4927-85e3-f3cceefe3a53)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53"
Oct 31 00:33:19.561464 ip-10-0-142-217 kubenswrapper[2578]: I1031 00:33:19.561393    2578 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:33:19.561565 ip-10-0-142-217 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 00:33:19.575087 ip-10-0-142-217 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 00:33:19.575260 ip-10-0-142-217 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 00:33:19.575309 ip-10-0-142-217 systemd[1]: kubelet.service: Consumed 2.979s CPU time.
Oct 31 00:33:19.588416 ip-10-0-142-217 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 00:33:19.672943 ip-10-0-142-217 kubenswrapper[4924]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:33:19.672943 ip-10-0-142-217 kubenswrapper[4924]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 00:33:19.672943 ip-10-0-142-217 kubenswrapper[4924]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:33:19.672943 ip-10-0-142-217 kubenswrapper[4924]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:33:19.675067 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.674976    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.675067 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675066    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.675067 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675070    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675074    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675077    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675080    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675083    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675085    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675088    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675091    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675094    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675096    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675099    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675101    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675104    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675107    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675109    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675112    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675115    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675117    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675120    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675123    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.675169 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675125    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675128    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675131    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675134    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675137    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675148    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675151    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675153    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675156    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675159    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675161    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675164    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675167    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675169    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675172    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675175    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675178    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675180    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675183    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.675661 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675187    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675189    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675192    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675195    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675197    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675200    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675203    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675206    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675209    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675211    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675214    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675217    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675219    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675222    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675225    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675227    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675230    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675234    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675238    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.676155 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675241    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675247    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675250    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675253    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675255    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675258    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675261    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675263    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675266    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675268    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675271    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675274    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675276    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675279    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675282    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675285    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675289    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675291    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675294    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675297    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.676660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675299    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675302    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675305    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675307    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675310    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675312    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675315    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675318    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675321    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675324    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675326    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675329    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675336    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675339    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675342    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675345    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675347    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675350    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675352    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675355    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.677215 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675357    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675360    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675362    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675366    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675369    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.675372    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676454    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676462    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676466    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676470    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676472    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676475    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676478    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676481    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676483    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676486    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676488    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676491    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676494    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676496    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.677719 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676499    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676501    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676504    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676507    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676509    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676513    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676515    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676518    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676520    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676523    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676525    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676528    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676531    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676533    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676536    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676539    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676541    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676544    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676547    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676550    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.678247 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676552    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676555    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676558    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676560    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676562    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676565    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676568    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676570    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676572    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676575    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676579    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676581    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676584    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676586    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676589    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676591    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676594    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676598    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676602    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.678726 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676606    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676608    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676611    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676613    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676619    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676622    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676625    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676629    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676632    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676635    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676639    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676642    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676645    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676647    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676650    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676653    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676655    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676658    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676661    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.679196 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676663    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676666    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676669    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676672    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676675    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676677    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676680    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676682    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676685    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676687    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676691    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676693    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676696    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676699    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676702    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676704    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676706    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676709    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676711    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.679668 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676714    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676716    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676719    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676721    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676724    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676727    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676730    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676733    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676735    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676737    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676740    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676758    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676761    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676764    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.676766    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677256    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677261    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677264    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677267    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.680140 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677269    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677272    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677275    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677277    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677280    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677283    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677286    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677290    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677293    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677296    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677298    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677301    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677304    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677306    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677309    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677311    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677314    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677317    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677319    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.680607 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677323    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677327    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677330    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677332    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677335    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677338    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677340    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677343    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677345    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677348    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677351    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677353    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677355    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677358    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677360    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677364    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677367    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677370    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677374    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677377    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.681088 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677380    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677383    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677385    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677388    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677390    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677393    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677396    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677398    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677401    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677403    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677406    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677409    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677412    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677414    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677417    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677420    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677423    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677425    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677428    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677431    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.681596 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677433    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677436    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677439    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677441    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677443    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677446    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677449    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677452    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677454    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677457    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677460    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677462    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677465    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677467    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677470    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677473    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677475    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677480    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677483    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677485    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.682094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677488    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677490    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677493    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677496    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677498    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677501    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677503    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677506    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677508    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677511    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677513    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677516    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677518    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677521    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677524    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677526    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677529    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677532    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677534    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677537    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.682577 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677540    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677542    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.677545    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677647    4924 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677659    4924 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677665    4924 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677670    4924 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677675    4924 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677679    4924 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677686    4924 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677690    4924 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677693    4924 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677696    4924 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677700    4924 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677703    4924 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677707    4924 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677710    4924 flags.go:64] FLAG: --cgroup-root=""
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677713    4924 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677716    4924 flags.go:64] FLAG: --client-ca-file=""
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677720    4924 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677723    4924 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677727    4924 flags.go:64] FLAG: --cluster-domain=""
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677731    4924 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677734    4924 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 00:33:19.683257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677738    4924 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677755    4924 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677762    4924 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677766    4924 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677770    4924 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677773    4924 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677776    4924 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677781    4924 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677785    4924 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677788    4924 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677791    4924 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677796    4924 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677799    4924 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677802    4924 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677805    4924 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677808    4924 flags.go:64] FLAG: --enable-server="true"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677811    4924 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677817    4924 flags.go:64] FLAG: --event-burst="100"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677820    4924 flags.go:64] FLAG: --event-qps="50"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677823    4924 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677826    4924 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677829    4924 flags.go:64] FLAG: --eviction-hard=""
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677834    4924 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677837    4924 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677840    4924 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 00:33:19.684091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677844    4924 flags.go:64] FLAG: --eviction-soft=""
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677847    4924 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677851    4924 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677854    4924 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677857    4924 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677860    4924 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677863    4924 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677866    4924 flags.go:64] FLAG: --feature-gates=""
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677871    4924 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677874    4924 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677877    4924 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677881    4924 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677884    4924 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677887    4924 flags.go:64] FLAG: --help="false"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677891    4924 flags.go:64] FLAG: --hostname-override="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677895    4924 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677898    4924 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677901    4924 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677904    4924 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677908    4924 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677911    4924 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677914    4924 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677917    4924 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 00:33:19.684768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677920    4924 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677925    4924 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677929    4924 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677932    4924 flags.go:64] FLAG: --kube-reserved=""
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677935    4924 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677937    4924 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677940    4924 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677943    4924 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677946    4924 flags.go:64] FLAG: --lock-file=""
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677949    4924 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677952    4924 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677955    4924 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677961    4924 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677964    4924 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677967    4924 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677970    4924 flags.go:64] FLAG: --logging-format="text"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677973    4924 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677976    4924 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677979    4924 flags.go:64] FLAG: --manifest-url=""
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677982    4924 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677987    4924 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677990    4924 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677993    4924 flags.go:64] FLAG: --max-pods="110"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.677998    4924 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678001    4924 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 00:33:19.685340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678004    4924 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678007    4924 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678010    4924 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678013    4924 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678016    4924 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678023    4924 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678026    4924 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678029    4924 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678034    4924 flags.go:64] FLAG: --pod-cidr=""
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678038    4924 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678041    4924 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678044    4924 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678047    4924 flags.go:64] FLAG: --port="10250"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678049    4924 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678053    4924 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-05521f2a842664096"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678056    4924 flags.go:64] FLAG: --qos-reserved=""
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678059    4924 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678062    4924 flags.go:64] FLAG: --register-node="true"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678065    4924 flags.go:64] FLAG: --register-with-taints=""
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678069    4924 flags.go:64] FLAG: --registry-burst="10"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678072    4924 flags.go:64] FLAG: --registry-qps="5"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678075    4924 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678078    4924 flags.go:64] FLAG: --reserved-memory=""
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678082    4924 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678085    4924 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678088    4924 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 00:33:19.686055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678091    4924 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678093    4924 flags.go:64] FLAG: --runonce="false"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678097    4924 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678100    4924 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678105    4924 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678108    4924 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678111    4924 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678114    4924 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678117    4924 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678120    4924 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678123    4924 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678125    4924 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678128    4924 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678131    4924 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678135    4924 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678138    4924 flags.go:64] FLAG: --system-cgroups=""
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678142    4924 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678147    4924 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678150    4924 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678153    4924 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678158    4924 flags.go:64] FLAG: --tls-min-version=""
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678161    4924 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678163    4924 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678166    4924 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678169    4924 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 00:33:19.686681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678173    4924 flags.go:64] FLAG: --v="2"
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678177    4924 flags.go:64] FLAG: --version="false"
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678181    4924 flags.go:64] FLAG: --vmodule=""
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678185    4924 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678188    4924 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678321    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678325    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678328    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678331    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678334    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678337    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678341    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678344    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678346    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678349    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678352    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678356    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678360    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678363    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678365    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678368    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.687309 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678371    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678374    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678376    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678379    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678382    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678385    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678387    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678390    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678392    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678395    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678397    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678400    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678402    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678404    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678407    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678410    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678412    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678414    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678417    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678419    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.687852 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678422    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678424    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678429    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678431    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678434    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678436    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678439    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678441    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678444    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678446    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678449    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678451    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678454    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678457    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678459    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678462    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678465    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678468    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678471    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678474    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.688335 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678477    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678479    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678482    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678485    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678487    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678490    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678493    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678496    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678498    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678501    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678503    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678506    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678508    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678511    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678515    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678517    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678520    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678522    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678525    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.688843 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678527    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678530    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678532    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678535    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678538    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678541    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678543    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678546    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678548    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678551    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678554    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678556    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678559    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678562    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678564    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678567    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678570    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678572    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678575    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.689353 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678577    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678580    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678582    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678584    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678587    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678591    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678595    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678597    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678601    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678604    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678607    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678609    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.678635    4924 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678671    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678676    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.689984 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678680    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678683    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678686    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678689    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678691    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678694    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678697    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678699    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678703    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678707    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678710    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678713    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678716    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678718    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678721    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678724    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678726    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678729    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678732    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678736    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.690392 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678738    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678741    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678759    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678762    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678765    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678769    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678772    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678775    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678796    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678799    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678803    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678806    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678809    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678813    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678816    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678819    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678821    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678824    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678826    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678829    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.691216 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678832    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678834    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678838    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678840    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678843    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678845    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678848    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678851    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678853    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678855    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678858    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678861    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678863    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678866    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678869    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678872    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678875    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678881    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678884    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678886    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.692094 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678889    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678892    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678894    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678897    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678899    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678903    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678906    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678909    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678911    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678914    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678917    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678919    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678922    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678924    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678927    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678930    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678933    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678935    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678938    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.692641 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678940    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678942    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678945    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678948    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678950    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678953    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678955    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678958    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678960    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678963    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678967    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678970    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678972    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678975    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678977    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678980    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678982    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678986    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.693148 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678991    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678994    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678996    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.678999    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.679002    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.679006    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.679009    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.684323    4924 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.684342    4924 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.684367    4924 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.684372    4924 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684433    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684439    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684442    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684445    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684448    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684451    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684454    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684457    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684460    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.693592 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684462    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684465    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684468    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684470    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684473    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684476    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684479    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684482    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684485    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684488    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684492    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684494    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684497    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684499    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684502    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684505    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684507    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684510    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684514    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.694106 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684517    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684521    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684524    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684527    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684530    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684533    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684535    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684537    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684540    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684543    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684545    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684548    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684550    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684553    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684556    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684558    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684561    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684564    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684566    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.694560 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684569    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684572    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684575    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684577    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684581    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684583    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684586    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684588    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684591    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684594    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684596    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684599    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684601    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684604    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684606    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684609    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684612    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684614    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684617    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684619    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.695042 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684622    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684624    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684627    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684629    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684631    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684634    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684636    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684639    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684642    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684644    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684646    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684649    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684659    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684662    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684664    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684667    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684669    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684672    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684675    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.695516 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684677    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684680    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684682    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684685    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684688    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684690    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684693    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684695    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684698    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684701    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684703    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684706    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684710    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684714    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684717    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684720    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684723    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684726    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684728    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.695981 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684731    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.684780    4924 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684921    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684926    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684929    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684932    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684935    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684944    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684947    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684949    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684952    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684955    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684958    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684960    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684963    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684966    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.696478 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684969    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684971    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684974    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684976    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684979    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684981    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684983    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684986    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684988    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684991    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684994    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684996    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.684999    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685002    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685004    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685008    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685011    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685013    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685017    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.696939 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685020    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685023    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685026    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685028    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685031    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685039    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685042    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685044    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685047    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685049    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685051    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685054    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685057    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685059    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685062    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685064    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685067    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685070    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685072    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685075    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.697394 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685077    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685080    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685082    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685085    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685087    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685090    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685093    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685096    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685098    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685101    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685103    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685106    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685109    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685112    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685114    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685117    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685119    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685122    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685130    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685133    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.697881 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685136    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685138    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685141    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685143    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685146    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685148    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685151    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685153    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685155    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685158    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685161    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685164    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685166    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685169    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685172    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685174    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685177    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685180    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685183    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.698378 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685186    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685188    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685190    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685193    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685196    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685198    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685203    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685207    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685211    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685214    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685217    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685220    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685230    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685234    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.685257    4924 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:33:19.698911 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685290    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685294    4924 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685297    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685300    4924 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685303    4924 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685305    4924 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685308    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685311    4924 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685314    4924 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685317    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685319    4924 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685322    4924 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685324    4924 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685327    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685330    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685333    4924 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685335    4924 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685338    4924 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685340    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685343    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:33:19.699305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685346    4924 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685349    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685351    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685354    4924 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685358    4924 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685361    4924 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685364    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685366    4924 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685369    4924 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685372    4924 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685380    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685383    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685387    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685390    4924 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685392    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685395    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685397    4924 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685400    4924 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:33:19.699803 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685403    4924 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685406    4924 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685410    4924 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685413    4924 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685416    4924 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685418    4924 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685421    4924 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685424    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685427    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685430    4924 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685432    4924 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685435    4924 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685437    4924 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685440    4924 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685442    4924 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685445    4924 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685447    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685450    4924 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685454    4924 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:33:19.700242 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685457    4924 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685460    4924 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685462    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685465    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685468    4924 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685470    4924 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685479    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685484    4924 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685486    4924 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685489    4924 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685492    4924 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685494    4924 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685497    4924 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685499    4924 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685502    4924 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685504    4924 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685507    4924 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685509    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685512    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:33:19.700794 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685515    4924 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685517    4924 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685522    4924 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685525    4924 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685528    4924 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685531    4924 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685534    4924 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685538    4924 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685540    4924 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685543    4924 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685546    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685548    4924 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685553    4924 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685555    4924 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685558    4924 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685561    4924 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685563    4924 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685566    4924 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685568    4924 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685571    4924 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:33:19.701533 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685588    4924 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685592    4924 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685594    4924 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685597    4924 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685600    4924 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685602    4924 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685605    4924 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685608    4924 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685610    4924 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:19.685613    4924 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.685690    4924 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.686699    4924 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.686768    4924 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.686986    4924 server.go:1014] "Starting client certificate rotation"
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.687108    4924 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.687161    4924 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 00:27:39 +0000 UTC" deadline="2028-05-29 06:33:56.592975137 +0000 UTC"
Oct 31 00:33:19.702337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.687198    4924 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13830h0m36.905780158s"
Oct 31 00:33:19.702870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.687548    4924 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:33:19.702870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.688724    4924 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:33:19.702870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.689955    4924 log.go:25] "Validated CRI v1 runtime API"
Oct 31 00:33:19.702870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.692208    4924 log.go:25] "Validated CRI v1 image API"
Oct 31 00:33:19.702870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.692918    4924 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 00:33:19.702870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.694801    4924 fs.go:135] Filesystem UUIDs: map[08ca7b1e-a9f1-45f9-9e06-617f03eaae71:/dev/nvme0n1p3 77b26557-0233-42d6-aa53-6281f9eca005:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 31 00:33:19.703115 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.694822    4924 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6380445fd64913146f6ba600aee8e5eacf66c16176567efa266f7e325c0e1c7c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6380445fd64913146f6ba600aee8e5eacf66c16176567efa266f7e325c0e1c7c/userdata/shm major:0 minor:298 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7/userdata/shm major:0 minor:73 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/df6c1e3693d868ee10ceccaaaf860d2ebbc5a45483ec0fd3692ab1e68f900278/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/df6c1e3693d868ee10ceccaaaf860d2ebbc5a45483ec0fd3692ab1e68f900278/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e84748c0c29c4be6338fb082c311df6f65e039aaeb3fed3c5f62225fe9fcb1f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e84748c0c29c4be6338fb082c311df6f65e039aaeb3fed3c5f62225fe9fcb1f5/userdata/shm major:0 minor:317 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4/userdata/shm major:0 minor:289 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/007129ea-79e7-4781-a23c-8491d433c8a9/volumes/kubernetes.io~projected/kube-api-access-xglkq:{mountpoint:/var/lib/kubelet/pods/007129ea-79e7-4781-a23c-8491d433c8a9/volumes/kubernetes.io~projected/kube-api-access-xglkq major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03f1dde4-3764-4c15-821f-54ca5cd3b47f/volumes/kubernetes.io~projected/kube-api-access-f6hhh:{mountpoint:/var/lib/kubelet/pods/03f1dde4-3764-4c15-821f-54ca5cd3b47f/volumes/kubernetes.io~projected/kube-api-access-f6hhh major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/041328ee-8f09-4927-85e3-f3cceefe3a53/volumes/kubernetes.io~projected/kube-api-access-bb9b9:{mountpoint:/var/lib/kubelet/pods/041328ee-8f09-4927-85e3-f3cceefe3a53/volumes/kubernetes.io~projected/kube-api-access-bb9b9 major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/041328ee-8f09-4927-85e3-f3cceefe3a53/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/041328ee-8f09-4927-85e3-f3cceefe3a53/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15983cd6-0227-4191-8770-4fd87b6a52b2/volumes/kubernetes.io~projected/kube-api-access-ck9d4:{mountpoint:/var/lib/kubelet/pods/15983cd6-0227-4191-8770-4fd87b6a52b2/volumes/kubernetes.io~projected/kube-api-access-ck9d4 major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1dabbb8b-9e70-4c1c-92d7-04d112150395/volumes/kubernetes.io~projected/kube-api-access-qcpnr:{mountpoint:/var/lib/kubelet/pods/1dabbb8b-9e70-4c1c-92d7-04d112150395/volumes/kubernetes.io~projected/kube-api-access-qcpnr major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f593360-052d-45fe-8838-7395a5d16b21/volumes/kubernetes.io~projected/kube-api-access-27m64:{mountpoint:/var/lib/kubelet/pods/2f593360-052d-45fe-8838-7395a5d16b21/volumes/kubernetes.io~projected/kube-api-access-27m64 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f593360-052d-45fe-8838-7395a5d16b21/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/2f593360-052d-45fe-8838-7395a5d16b21/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~projected/kube-api-access-ss65p:{mountpoint:/var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~projected/kube-api-access-ss65p major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~secret/default-certificate major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~secret/stats-auth major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38dad442-b12d-4dc8-9f95-1471ff276929/volumes/kubernetes.io~projected/kube-api-access-fkv5h:{mountpoint:/var/lib/kubelet/pods/38dad442-b12d-4dc8-9f95-1471ff276929/volumes/kubernetes.io~projected/kube-api-access-fkv5h major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/44d79171-f637-4834-bfaa-bd57932eb5b4/volumes/kubernetes.io~projected/kube-api-access-j5h6s:{mountpoint:/var/lib/kubelet/pods/44d79171-f637-4834-bfaa-bd57932eb5b4/volumes/kubernetes.io~projected/kube-api-access-j5h6s major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c562d34-6b68-4800-a5a6-5c9e204a6050/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/4c562d34-6b68-4800-a5a6-5c9e204a6050/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:271 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff/volumes/kubernetes.io~projected/kube-api-access-n9fnb:{mountpoint:/var/lib/kubelet/pods/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff/volumes/kubernetes.io~projected/kube-api-access-n9fnb major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/542f4323-b31e-4491-9b6e-02a1916798ab/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/542f4323-b31e-4491-9b6e-02a1916798ab/volumes/kubernetes.io~secret/agent-certs major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65327963-1791-42b7-a9a7-4e77b700ed5a/volumes/kubernetes.io~projected/kube-api-access-5zhwq:{mountpoint:/var/lib/kubelet/pods/65327963-1791-42b7-a9a7-4e77b700ed5a/volumes/kubernetes.io~projected/kube-api-access-5zhwq major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volumes/kubernetes.io~projected/kube-api-access-zpqbn:{mountpoint:/var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volumes/kubernetes.io~projected/kube-api-access-zpqbn major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69a61c32-b459-46fa-a720-80637d99c7ae/volumes/kubernetes.io~projected/kube-api-access-8jx9h:{mountpoint:/var/lib/kubelet/pods/69a61c32-b459-46fa-a720-80637d99c7ae/volumes/kubernetes.io~projected/kube-api-access-8jx9h major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/791da7b8-431d-4ee0-8bfb-04b22f6d49c8/volumes/kubernetes.io~projected/kube-api-access-kdptw:{mountpoint:/var/lib/kubelet/pods/791da7b8-431d-4ee0-8bfb-04b22f6d49c8/volumes/kubernetes.io~projected/kube-api-access-kdptw major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c58bcb9-e087-48f9-91a7-cecacedd8242/volumes/kubernetes.io~projected/kube-api-access-rmdf6:{mountpoint:/var/lib/kubelet/pods/7c58bcb9-e087-48f9-91a7-cecacedd8242/volumes/kubernetes.io~projected/kube-api-access-rmdf6 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c58bcb9-e087-48f9-91a7-cecacedd8242/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7c58bcb9-e087-48f9-91a7-cecacedd8242/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/volumes/kubernetes.io~projected/kube-api-access-4t8cq:{mountpoint:/var/lib/kubelet/pods/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/volumes/kubernetes.io~projected/kube-api-access-4t8cq major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/volumes/kubernetes.io~secret/cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9557d837-f593-4d62-ad04-07b2c0b77987/volumes/kubernetes.io~projected/kube-api-access-hgzq2:{mountpoint:/var/lib/kubelet/pods/9557d837-f593-4d62-ad04-07b2c0b77987/volumes/kubernetes.io~projected/kube-api-access-hgzq2 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/995d046f-21e7-4152-bdf2-d0954c8ff37f/volumes/kubernetes.io~projected/kube-api-access-pqmwt:{mountpoint:/var/lib/kubelet/pods/995d046f-21e7-4152-bdf2-d0954c8ff37f/volumes/kubernetes.io~projected/kube-api-access-pqmwt major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~projected/kube-api-access-t2t9g:{mountpoint:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~projected/kube-api-access-t2t9g major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc/volumes/kubernetes.io~projected/kube-api-access-s8mpf:{mountpoint:/var/lib/kubelet/pods/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc/volumes/kubernetes.io~projected/kube-api-access-s8mpf major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab123256-37b2-4cf7-961b-38c7a0e3be21/volumes/kubernetes.io~projected/kube-api-access-b4smf:{mountpoint:/var/lib/kubelet/pods/ab123256-37b2-4cf7-961b-38c7a0e3be21/volumes/kubernetes.io~projected/kube-api-access-b4smf major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~projected/kube-api-access-thtgj:{mountpoint:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~projected/kube-api-access-thtgj major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb393179-7795-4aa8-bd36-538fe2821808/volumes/kubernetes.io~projected/kube-api-access-rnq72:{mountpoint:/var/lib/kubelet/pods/bb393179-7795-4aa8-bd36-538fe2821808/volumes/kubernetes.io~projected/kube-api-access-rnq72 major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~projected/kube-api-access-cj6gj:{mountpoint:/var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~projected/kube-api-access-cj6gj major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edb0ea49-807e-4139-a8bc-516f3b88b720/volumes/kubernetes.io~projected/kube-api-access-kwzqt:{mountpoint:/var/lib/kubelet/pods/edb0ea49-807e-4139-a8bc-516f3b88b720/volumes/kubernetes.io~projected/kube-api-access-kwzqt major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef90678c-925d-49bb-acd6-0d2861437916/volumes/kubernetes.io~projected/kube-api-access-sg77m:{mountpoint:/var/lib/kubelet/pods/ef90678c-925d-49bb-acd6-0d2861437916/volumes/kubernetes.io~projected/kube-api-access-sg77m major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fce6d408-267e-49f1-ac1e-68176943fbf9/volumes/kubernetes.io~projected/kube-api-access-9rct8:{mountpoint:/var/lib/kubelet/pods/fce6d408-267e-49f1-ac1e-68176943fbf9/volumes/kubernetes.io~projected/kube-api-access-9rct8 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff9c0bff-850c-4b57-b3fe-ea825f6d4767/volumes/kubernetes.io~projected/kube-api-access-kwdwd:{mountpoint:/var/lib/kubelet/pods/ff9c0bff-850c-4b57-b3fe-ea825f6d4767/volumes/kubernetes.io~projected/kube-api-access-kwdwd major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff9c0bff-850c-4b57-b3fe-ea825f6d4767/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/ff9c0bff-850c-4b57-b3fe-ea825f6d4767/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/e7f5a86760445833765e4fd812c39b986aa7e7c26aed871d175ac1b1ebdf1eba/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/f1face063af841c97bf3b2a2b6f07671a3a368300762fe7d446c663d84d5d90d/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/ddd847eeef867a4e804dfb4bfd2548f3553b3ef71ecc5042613e48ea2f341149/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/e6e44cc9f65d58b6a1749f917e8973ee799ce3a7e2d1dbe85cbacb32434be088/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/fdb5cfbd046c228f450bd7ec3d4c40f823d15c53bd32aed0345baa0d3d769f6b/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/4464bbb9670f938d52f6ec8925f81ef242dd8fd949acc3fd67a39e403bb96997/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/e550360d84ff445d50a7dd7a78e18147243489be17ec8815692e3731400f46c5/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/134243935b5b1dc629871b2036ab3eedf4c628c895fb428aa3d3279691076c65/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/ec6c5c30b2682fc2c1d333236cc78795ab591b4703d8396244b3183b5d19b78b/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/607808faf032d4458dd7f6420c9084ac6d86b2df241467643883b19697eba23b/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/9c1cd7dd963ec4971501bdf27a25e817950359b1bc9767914b76e0a1caa784aa/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-151:{mountpoint:/var/lib/containers/storage/overlay/0461f2f85330079ed79f7e067b72a69a98d84b881c274a704cd1480d85faa502/merged major:0 minor:151 fsType:overlay blockSize:0} overlay_0-153:{mountpoint:/var/lib/containers/storage/overlay/b12b090112659564c7c5c68d2056f36ad46077faf37bbbb66e930ee575f071d2/merged major:0 minor:153 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/f57cb210b2fec91f466cf624db8a5bafa1114a6d9dc385eac46b1311f73f84e0/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/2b831bd3f01e372d443904ca2a9d2306f5ecc1f92fd3045fd6c1ebab17bf0660/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/fb0a1884bbb452a1c8ff49eddb925f4823a30d1654d66e8e9801ff848a154c33/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/4862b8d1b9a5afedc40df40d6c94147f9717ebb7b72975bbc688509f9a25cc71/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/08e6704d761789b677759001f68244092b6d2648a6799870cbef898f20ad0e6d/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/ac0adbeb155f8c0324b179a46ee1015dfe4694b96e5c9fa7a0375d82a3a1a7a9/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/6d308e77184fb37bc55953568f32612b7050764cb802c4f59faea4c8fd8e5567/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/a2e205e4274e830cccf1e8b8c9e79c61cb7499a09a8323ae0422f96939876878/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/21fd654c46610a4929a589e09445cf33d9d8934bd35a389a6428ea266d04340b/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/fa3f8f4e0adca1fb2c764e20cb527387c62145eb0168b7b863f867b83f566591/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/4361b3d3ab9bf55dbfbcabe4ff53f51a41299ee26b3f0656b9b3f5a475b73dad/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/464599bffc8f6cdcec64bd90ac2c775a204c3ead22bb4f79d4c5cb59b94e019b/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/e9ef7a73b3d8bf564fbec1c68b43c69f72cef9c5c4c1a7edd504ee42fe4ccf02/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/1c944747221247cc35046484a2eba048c8118465a854548e97590ace2a394a21/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/ae44bd27974a640cf2af48a1c61d1162bdcea7c1f9655aa91c2cb1eacdba4dc1/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/4937c83bc125c2be976cd263bf9caa571b1988cefa32105e40059fe31407fcbd/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/6eb06142103f54965721bd11030b916dc11f06c6fc8db868c9e481449f8f6ed9/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/a27f93053d506dddef44fcc69a3b5867581efc9c7557484615b8501578332b8c/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/2a37505fba690f3099680fd08faee993a33c7d7cdbc9c85a45973063651ade66/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/0a9449be9196b1c10f201a57f0f09823a61282ccd2b53f38becc1c6da1c6160f/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/f5cc5106574b00961a9b10baecdeb463e010a26f6395622d866eede86de51d5c/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/c499f9766ceda10db4cea77e24cd8407d30667fd9b56a8f3f342f9df21e7424c/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/5cc36f44e7fa816cd6a611da69641bf900ed3faabc737b71fe3846d283a5a99d/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/8e2acc1854cc424d4d256f1cc5f91cce782ea8294063363af61f8af0bf4b3bd9/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/e0e3bf605379fa0e550065090aea60cad2dd6b8d3372fa89d3322857d0879ac8/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/837bf0257bb5495111e88e3e019432d4537d513a107f8df3d4a520b2bad95c07/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/5002e09478d57e19f6a59b0332ec28a8457a8f5017839f3fb629dd2635cc2514/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/ee1947a272f0c41a632367ba4c80b5e5cec729c1b87088f4338739867c49f86e/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/010bf5ba751b37e38a4476adf580defb380bb73ff95a0386f909fcea63b7c694/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/8fd600c068ed5047c8d5ac571c886dd097b1c59ac77867ad339988efcc376d0b/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/cea3192200bec22e15c687031aea6de34bc1224fbc17eb127fb9e999d8f072ca/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/70a7d2cfd1a7b77a42d7e060fb07adc7c180cc08efa46131a70effab5d28b01e/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/3f0e9f71a08aec69452e5a0922078b2142da828aedcd7e8c7b652a473f6ec77f/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/4596c45e14a2e511f5e175cb5afdd37ab6ed3c5de2568d0ce0603a55607d0350/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/5dea77c191f67ec5442448f7b4b339faebf95265e0c8eaa0ca7b4b7bc3c416dd/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/dda96ca375c5c63564e698898e6e5f4c1160a4702bc5d462c1fff4db6c54ba24/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-75:{mountpoint:/var/lib/containers/storage/overlay/cd2dda834f7854f88edf28d561994bf844eb25b44a0f79584fd3d5fd848c3fa8/merged major:0 minor:75 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/fd4ff46823df96b895f090d8edbc177209f05ae6bb2980c58525325702556f76/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/5461fd25c6d2a6c49dd67126e4582ea840aa017321209ca974311bd9c7f71124/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/4208d26abb48160b951dbc8797f80b0c77b4958dfe87236afdbf3f2d226cbd48/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/e76648db7adba9105483cdf8241fd42493ffeed3195f9723a7de1654a8aa5386/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ed6967c46da5bbba9a674a6b23a951804fe6b31187ec8a78af1758d2f7ad10b3/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/a966513ef10f87a7d482c6dd25f1216c39ec2c391e98df051194347cb150085c/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 31 00:33:19.705320 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.704893    4924 manager.go:217] Machine: {Timestamp:2026-10-31 00:33:19.704405861 +0000 UTC m=+0.075189641 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3067738 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24efd0f0ef3856c608c1c635d066b8 SystemUUID:ec24efd0-f0ef-3856-c608-c1c635d066b8 BootID:4545dc03-0757-48d9-ad7d-8da780e5676c Filesystems:[{Device:/run/containers/storage/overlay-containers/64f5645c2de00f49e399cd5714d2dc3bf5490b83022e42b43b1a971f6f00375e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1dabbb8b-9e70-4c1c-92d7-04d112150395/volumes/kubernetes.io~projected/kube-api-access-qcpnr DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fd104229de68d3aea3b857bf42ffc83b7565d272a8174f4acf390ec33f3b311e/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/volumes/kubernetes.io~projected/kube-api-access-4t8cq DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~projected/kube-api-access-cj6gj DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/041328ee-8f09-4927-85e3-f3cceefe3a53/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc/volumes/kubernetes.io~projected/kube-api-access-s8mpf DeviceMajor:0 DeviceMinor:310 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~projected/kube-api-access-ss65p DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/edb0ea49-807e-4139-a8bc-516f3b88b720/volumes/kubernetes.io~projected/kube-api-access-kwzqt DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~projected/kube-api-access-t2t9g DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/542f4323-b31e-4491-9b6e-02a1916798ab/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab123256-37b2-4cf7-961b-38c7a0e3be21/volumes/kubernetes.io~projected/kube-api-access-b4smf DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c27f2b51d705e10086b7b19215394cb761c9ea44b3241ece61bec5cf5221ea92/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ff9c0bff-850c-4b57-b3fe-ea825f6d4767/volumes/kubernetes.io~projected/kube-api-access-kwdwd DeviceMajor:0 DeviceMinor:216 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/670c1bdb65094fcdd326af6ed2b85b541f9c97149c42e038c6d4be55dc50c66e/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cbd984040293db2f39cad07356904bd7cb6e314009ef921df710f78e715b5439/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f33de09c368e59e7ce2b841730aefeb99d9248523a458d69265f44a5046913f4/userdata/shm DeviceMajor:0 DeviceMinor:289 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ff9c0bff-850c-4b57-b3fe-ea825f6d4767/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9836bf56b6e4bc04f69c5685403ca0a3dc33fc2a9931f9165e366382378db344/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e84748c0c29c4be6338fb082c311df6f65e039aaeb3fed3c5f62225fe9fcb1f5/userdata/shm DeviceMajor:0 DeviceMinor:317 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff/volumes/kubernetes.io~projected/kube-api-access-n9fnb DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b32722819a61c18e64ea96eb18ec50ffde4e38e8d42cf7289a593df64918c5d6/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/995d046f-21e7-4152-bdf2-d0954c8ff37f/volumes/kubernetes.io~projected/kube-api-access-pqmwt DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65327963-1791-42b7-a9a7-4e77b700ed5a/volumes/kubernetes.io~projected/kube-api-access-5zhwq DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-75 DeviceMajor:0 DeviceMinor:75 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5b571d80863f8a07374a8d14cfb3f3d8024c1c0f7a527e0911252a0ace559d8d/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6380445fd64913146f6ba600aee8e5eacf66c16176567efa266f7e325c0e1c7c/userdata/shm DeviceMajor:0 DeviceMinor:298 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9de808333f9cabefa750634130226b002b4892cb307b4c4d185db713410f52a1/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dfde58a0-def5-41fb-8f2d-e825a8ab34fe/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/03f1dde4-3764-4c15-821f-54ca5cd3b47f/volumes/kubernetes.io~projected/kube-api-access-f6hhh DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c58bcb9-e087-48f9-91a7-cecacedd8242/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2f593360-052d-45fe-8838-7395a5d16b21/volumes/kubernetes.io~projected/kube-api-access-27m64 DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15983cd6-0227-4191-8770-4fd87b6a52b2/volumes/kubernetes.io~projected/kube-api-access-ck9d4 DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4322fff50d0fe2033d7b6fa681e1b847bbebb68453cec365b401497c510ad891/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4c562d34-6b68-4800-a5a6-5c9e204a6050/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:271 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04cc79cbe5ed38abb89b74737d24da87649ead261e37a8da4cae7cbdc627b282/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/44d79171-f637-4834-bfaa-bd57932eb5b4/volumes/kubernetes.io~projected/kube-api-access-j5h6s DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/041328ee-8f09-4927-85e3-f3cceefe3a53/volumes/kubernetes.io~projected/kube-api-access-bb9b9 DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/007129ea-79e7-4781-a23c-8491d433c8a9/volumes/kubernetes.io~projected/kube-api-access-xglkq DeviceMajor:0 DeviceMinor:292 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2f593360-052d-45fe-8838-7395a5d16b21/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e60c0c2d5c42c7e0e62e760b6cd6635e3becbe13a0eb0fcff46872b4f7d6d3b7/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b78c932070a4304990d184aed4f9a404d0453217f137c75e2f1cdd490708d28/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-153 DeviceMajor:0 DeviceMinor:153 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3570c076-c980-454d-9b4d-4fbcc88953e0/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/791da7b8-431d-4ee0-8bfb-04b22f6d49c8/volumes/kubernetes.io~projected/kube-api-access-kdptw DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/38dad442-b12d-4dc8-9f95-1471ff276929/volumes/kubernetes.io~projected/kube-api-access-fkv5h DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ef90678c-925d-49bb-acd6-0d2861437916/volumes/kubernetes.io~projected/kube-api-access-sg77m DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/69a61c32-b459-46fa-a720-80637d99c7ae/volumes/kubernetes.io~projected/kube-api-access-8jx9h DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6643e3d6-52ac-4e9d-af24-8a18f92b3748/volumes/kubernetes.io~projected/kube-api-access-zpqbn DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fce6d408-267e-49f1-ac1e-68176943fbf9/volumes/kubernetes.io~projected/kube-api-access-9rct8 DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-151 DeviceMajor:0 DeviceMinor:151 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d2f07b4e62a469dbd883f7654b9bc1b501518ea74c519047eb9ba021f3cc65e7/userdata/shm DeviceMajor:0 DeviceMinor:73 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7c58bcb9-e087-48f9-91a7-cecacedd8242/volumes/kubernetes.io~projected/kube-api-access-rmdf6 DeviceMajor:0 DeviceMinor:218 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d325374aafb15f737276f2526762497782577b8ee727e219a5b36aa6078b5ed/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/32cc6fa8f4393cd8c572d30c95b584431893d0cb16798d37000202e2ff570ed0/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes/kubernetes.io~projected/kube-api-access-thtgj DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/df6c1e3693d868ee10ceccaaaf860d2ebbc5a45483ec0fd3692ab1e68f900278/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0db673b45d4e388d8df44c877ceea8b5f2dec44acbe0b3762715cc75789ba0d2/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bb393179-7795-4aa8-bd36-538fe2821808/volumes/kubernetes.io~projected/kube-api-access-rnq72 DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9557d837-f593-4d62-ad04-07b2c0b77987/volumes/kubernetes.io~projected/kube-api-access-hgzq2 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0db673b45d4e388 MacAddress:82:28:25:44:61:13 Speed:10000 Mtu:8901} {Name:1b78c932070a430 MacAddress:ce:57:6c:9a:b3:cf Speed:10000 Mtu:8901} {Name:32cc6fa8f4393cd MacAddress:ee:ee:da:1f:f6:20 Speed:10000 Mtu:8901} {Name:4322fff50d0fe20 MacAddress:66:00:d8:32:d5:9e Speed:10000 Mtu:8901} {Name:5b571d80863f8a0 MacAddress:8e:6d:05:b4:3b:53 Speed:10000 Mtu:8901} {Name:6380445fd649131 MacAddress:2a:24:5b:01:9a:49 Speed:10000 Mtu:8901} {Name:670c1bdb65094fc MacAddress:e6:85:c1:88:4d:18 Speed:10000 Mtu:8901} {Name:7d325374aafb15f MacAddress:6a:5a:22:b9:bc:45 Speed:10000 Mtu:8901} {Name:9de808333f9cabe MacAddress:66:0a:26:3a:cc:39 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c3:6e:a1:09 Speed:0 Mtu:9001} {Name:br-int MacAddress:76:ff:af:f9:b7:96 Speed:0 Mtu:8901} {Name:df6c1e3693d868e MacAddress:22:93:90:a2:c7:88 Speed:10000 Mtu:8901} {Name:e60c0c2d5c42c7e MacAddress:8e:84:3d:8a:1b:98 Speed:10000 Mtu:8901} {Name:e84748c0c29c4be MacAddress:2a:34:ce:ef:e2:59 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c3:6e:a1:09 Speed:0 Mtu:9001} {Name:f33de09c368e59e MacAddress:96:af:80:98:25:c3 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:6e:71:e6:d5:e0:c5 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:66:19:e6:84:39:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 00:33:19.705527 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705325    4924 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 00:33:19.705527 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705374    4924 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 00:33:19.705770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705721    4924 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 00:33:19.705899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705771    4924 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-217.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 00:33:19.705940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705901    4924 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 00:33:19.705940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705909    4924 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 00:33:19.705940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705919    4924 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:33:19.705940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705932    4924 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:33:19.706043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.705952    4924 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 00:33:19.706146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706137    4924 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 00:33:19.706260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706251    4924 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 00:33:19.706295 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706286    4924 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 00:33:19.706328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706299    4924 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 00:33:19.706328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706310    4924 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 00:33:19.706328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706319    4924 kubelet.go:405] "Adding apiserver pod source"
Oct 31 00:33:19.706328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.706327    4924 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 00:33:19.707364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.707347    4924 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 00:33:19.707364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.707365    4924 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:33:19.707962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.707944    4924 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 00:33:19.710696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.710681    4924 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 00:33:19.710696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.710695    4924 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:33:19.710823 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.710813    4924 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 00:33:19.711024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711014    4924 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 00:33:19.711053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711043    4924 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 00:33:19.711145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711136    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 00:33:19.711178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711150    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 00:33:19.711178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711159    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 00:33:19.711178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711169    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 00:33:19.711178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711178    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711187    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711194    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711203    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711211    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711219    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711231    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 00:33:19.711280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711245    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 00:33:19.711455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711293    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 00:33:19.711455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.711299    4924 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 00:33:19.714916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.714899    4924 server.go:1263] "Started kubelet"
Oct 31 00:33:19.715057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.714988    4924 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 00:33:19.715116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.715082    4924 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 00:33:19.715452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.715370    4924 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 00:33:19.715566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.714975    4924 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 00:33:19.715542 ip-10-0-142-217 systemd[1]: Started Kubernetes Kubelet.
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.719829    4924 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.720296    4924 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.722855    4924 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.723841    4924 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:27:40 +0000 UTC" deadline="2028-04-29 08:57:52.435252257 +0000 UTC"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.723880    4924 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13112h24m32.711375375s"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.724440    4924 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.724455    4924 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:19.724489    4924 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-217.us-east-2.compute.internal\" not found"
Oct 31 00:33:19.724631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.724569    4924 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 00:33:19.725154 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.725113    4924 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:33:19.729240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.728592    4924 factory.go:55] Registering systemd factory
Oct 31 00:33:19.729240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.728612    4924 factory.go:223] Registration of the systemd container factory successfully
Oct 31 00:33:19.729240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.728977    4924 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:33:19.729240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729078    4924 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:33:19.729240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729222    4924 factory.go:153] Registering CRI-O factory
Oct 31 00:33:19.729240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729235    4924 factory.go:223] Registration of the crio container factory successfully
Oct 31 00:33:19.729608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729287    4924 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 00:33:19.729608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729305    4924 factory.go:103] Registering Raw factory
Oct 31 00:33:19.729608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729319    4924 manager.go:1199] Started watching for new ooms in manager
Oct 31 00:33:19.729608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.729610    4924 manager.go:319] Starting recovery of all containers
Oct 31 00:33:19.730988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.730939    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542f4323-b31e-4491-9b6e-02a1916798ab" volumeName="kubernetes.io/secret/542f4323-b31e-4491-9b6e-02a1916798ab-agent-certs" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.730991    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9557d837-f593-4d62-ad04-07b2c0b77987" volumeName="kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731007    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="995d046f-21e7-4152-bdf2-d0954c8ff37f" volumeName="kubernetes.io/projected/995d046f-21e7-4152-bdf2-d0954c8ff37f-kube-api-access-pqmwt" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731020    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731034    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc" volumeName="kubernetes.io/empty-dir/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-data-volume" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731047    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731059    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4e15376-708a-442b-9439-8bec92eae34c" volumeName="kubernetes.io/configmap/f4e15376-708a-442b-9439-8bec92eae34c-nginx-conf" seLinuxMountContext=""
Oct 31 00:33:19.731116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731071    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edb0ea49-807e-4139-a8bc-516f3b88b720" volumeName="kubernetes.io/configmap/edb0ea49-807e-4139-a8bc-516f3b88b720-iptables-alerter-script" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731335    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" volumeName="kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-tmp" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731356    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" volumeName="kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-snapshots" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731370    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="041328ee-8f09-4927-85e3-f3cceefe3a53" volumeName="kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-config" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731383    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1dabbb8b-9e70-4c1c-92d7-04d112150395" volumeName="kubernetes.io/configmap/1dabbb8b-9e70-4c1c-92d7-04d112150395-telemetry-config" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731395    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f593360-052d-45fe-8838-7395a5d16b21" volumeName="kubernetes.io/projected/2f593360-052d-45fe-8838-7395a5d16b21-kube-api-access-27m64" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731408    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6643e3d6-52ac-4e9d-af24-8a18f92b3748" volumeName="kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-config" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731422    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69a61c32-b459-46fa-a720-80637d99c7ae" volumeName="kubernetes.io/projected/69a61c32-b459-46fa-a720-80637d99c7ae-kube-api-access-8jx9h" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731434    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee" volumeName="kubernetes.io/secret/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-cert" seLinuxMountContext=""
Oct 31 00:33:19.731446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731447    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731459    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc" volumeName="kubernetes.io/projected/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-api-access-s8mpf" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731472    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f593360-052d-45fe-8838-7395a5d16b21" volumeName="kubernetes.io/configmap/2f593360-052d-45fe-8838-7395a5d16b21-config" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731484    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65327963-1791-42b7-a9a7-4e77b700ed5a" volumeName="kubernetes.io/projected/65327963-1791-42b7-a9a7-4e77b700ed5a-kube-api-access-5zhwq" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731495    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee" volumeName="kubernetes.io/projected/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-kube-api-access-4t8cq" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731519    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731533    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab123256-37b2-4cf7-961b-38c7a0e3be21" volumeName="kubernetes.io/projected/ab123256-37b2-4cf7-961b-38c7a0e3be21-kube-api-access-b4smf" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731544    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731557    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731569    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce6d408-267e-49f1-ac1e-68176943fbf9" volumeName="kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-cni-binary-copy" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731580    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9557d837-f593-4d62-ad04-07b2c0b77987" volumeName="kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-binary-copy" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731592    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731621    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb393179-7795-4aa8-bd36-538fe2821808" volumeName="kubernetes.io/projected/bb393179-7795-4aa8-bd36-538fe2821808-kube-api-access-rnq72" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731634    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfde58a0-def5-41fb-8f2d-e825a8ab34fe" volumeName="kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-tuned" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731646    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce6d408-267e-49f1-ac1e-68176943fbf9" volumeName="kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-daemon-config" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731665    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="007129ea-79e7-4781-a23c-8491d433c8a9" volumeName="kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731679    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f593360-052d-45fe-8838-7395a5d16b21" volumeName="kubernetes.io/secret/2f593360-052d-45fe-8838-7395a5d16b21-serving-cert" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731692    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6643e3d6-52ac-4e9d-af24-8a18f92b3748" volumeName="kubernetes.io/secret/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731705    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731725    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef90678c-925d-49bb-acd6-0d2861437916" volumeName="kubernetes.io/projected/ef90678c-925d-49bb-acd6-0d2861437916-kube-api-access-sg77m" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731737    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44d79171-f637-4834-bfaa-bd57932eb5b4" volumeName="kubernetes.io/projected/44d79171-f637-4834-bfaa-bd57932eb5b4-kube-api-access-j5h6s" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731765    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6643e3d6-52ac-4e9d-af24-8a18f92b3748" volumeName="kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-env-overrides" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731777    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="791da7b8-431d-4ee0-8bfb-04b22f6d49c8" volumeName="kubernetes.io/configmap/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-config" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731789    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9557d837-f593-4d62-ad04-07b2c0b77987" volumeName="kubernetes.io/projected/9557d837-f593-4d62-ad04-07b2c0b77987-kube-api-access-hgzq2" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731803    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731815    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" volumeName="kubernetes.io/projected/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-kube-api-access-kwdwd" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731827    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="041328ee-8f09-4927-85e3-f3cceefe3a53" volumeName="kubernetes.io/secret/041328ee-8f09-4927-85e3-f3cceefe3a53-serving-cert" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731841    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6643e3d6-52ac-4e9d-af24-8a18f92b3748" volumeName="kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731856    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="791da7b8-431d-4ee0-8bfb-04b22f6d49c8" volumeName="kubernetes.io/projected/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-kube-api-access-kdptw" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731868    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c58bcb9-e087-48f9-91a7-cecacedd8242" volumeName="kubernetes.io/secret/7c58bcb9-e087-48f9-91a7-cecacedd8242-serving-cert" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731881    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef90678c-925d-49bb-acd6-0d2861437916" volumeName="kubernetes.io/configmap/ef90678c-925d-49bb-acd6-0d2861437916-config-volume" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731893    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c7b1d82-f9da-4de8-bb8f-f8e76a938aff" volumeName="kubernetes.io/projected/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-kube-api-access-n9fnb" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731907    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731920    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731931    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfde58a0-def5-41fb-8f2d-e825a8ab34fe" volumeName="kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-tmp" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731944    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" volumeName="kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731959    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03f1dde4-3764-4c15-821f-54ca5cd3b47f" volumeName="kubernetes.io/projected/03f1dde4-3764-4c15-821f-54ca5cd3b47f-kube-api-access-f6hhh" seLinuxMountContext=""
Oct 31 00:33:19.731950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731971    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1dabbb8b-9e70-4c1c-92d7-04d112150395" volumeName="kubernetes.io/projected/1dabbb8b-9e70-4c1c-92d7-04d112150395-kube-api-access-qcpnr" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731984    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3570c076-c980-454d-9b4d-4fbcc88953e0" volumeName="kubernetes.io/projected/3570c076-c980-454d-9b4d-4fbcc88953e0-kube-api-access-ss65p" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.731998    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c562d34-6b68-4800-a5a6-5c9e204a6050" volumeName="kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732020    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b395282f-6fab-4ee4-b581-c5194700dc6b" volumeName="kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732032    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce6d408-267e-49f1-ac1e-68176943fbf9" volumeName="kubernetes.io/projected/fce6d408-267e-49f1-ac1e-68176943fbf9-kube-api-access-9rct8" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732045    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15983cd6-0227-4191-8770-4fd87b6a52b2" volumeName="kubernetes.io/empty-dir/15983cd6-0227-4191-8770-4fd87b6a52b2-tmp-dir" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732060    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542f4323-b31e-4491-9b6e-02a1916798ab" volumeName="kubernetes.io/configmap/542f4323-b31e-4491-9b6e-02a1916798ab-konnectivity-ca" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732072    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732091    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edb0ea49-807e-4139-a8bc-516f3b88b720" volumeName="kubernetes.io/projected/edb0ea49-807e-4139-a8bc-516f3b88b720-kube-api-access-kwzqt" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732104    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef90678c-925d-49bb-acd6-0d2861437916" volumeName="kubernetes.io/empty-dir/ef90678c-925d-49bb-acd6-0d2861437916-tmp-dir" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732116    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="041328ee-8f09-4927-85e3-f3cceefe3a53" volumeName="kubernetes.io/projected/041328ee-8f09-4927-85e3-f3cceefe3a53-kube-api-access-bb9b9" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732131    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c58bcb9-e087-48f9-91a7-cecacedd8242" volumeName="kubernetes.io/projected/7c58bcb9-e087-48f9-91a7-cecacedd8242-kube-api-access-rmdf6" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732143    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9db3fd71-290b-4156-afe5-af15a922b238" volumeName="kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732155    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15983cd6-0227-4191-8770-4fd87b6a52b2" volumeName="kubernetes.io/projected/15983cd6-0227-4191-8770-4fd87b6a52b2-kube-api-access-ck9d4" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732167    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6643e3d6-52ac-4e9d-af24-8a18f92b3748" volumeName="kubernetes.io/projected/6643e3d6-52ac-4e9d-af24-8a18f92b3748-kube-api-access-zpqbn" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732180    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="041328ee-8f09-4927-85e3-f3cceefe3a53" volumeName="kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-trusted-ca" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732194    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9557d837-f593-4d62-ad04-07b2c0b77987" volumeName="kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732208    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfde58a0-def5-41fb-8f2d-e825a8ab34fe" volumeName="kubernetes.io/projected/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-kube-api-access-cj6gj" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732221    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" volumeName="kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-service-ca-bundle" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732234    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3570c076-c980-454d-9b4d-4fbcc88953e0" volumeName="kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-stats-auth" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732246    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38dad442-b12d-4dc8-9f95-1471ff276929" volumeName="kubernetes.io/projected/38dad442-b12d-4dc8-9f95-1471ff276929-kube-api-access-fkv5h" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732258    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c7b1d82-f9da-4de8-bb8f-f8e76a938aff" volumeName="kubernetes.io/configmap/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-serviceca" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732271    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c58bcb9-e087-48f9-91a7-cecacedd8242" volumeName="kubernetes.io/configmap/7c58bcb9-e087-48f9-91a7-cecacedd8242-config" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732284    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" volumeName="kubernetes.io/secret/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-serving-cert" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732297    4924 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3570c076-c980-454d-9b4d-4fbcc88953e0" volumeName="kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-default-certificate" seLinuxMountContext=""
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732309    4924 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 00:33:19.733664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.732317    4924 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 00:33:19.736839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.736815    4924 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 00:33:19.747729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.747681    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/1.log"
Oct 31 00:33:19.748128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.748108    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/1.log"
Oct 31 00:33:19.754364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.754296    4924 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 00:33:19.755897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.755872    4924 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 00:33:19.755897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.755897    4924 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 00:33:19.756071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.755920    4924 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 00:33:19.756071 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:19.755964    4924 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 00:33:19.759970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.759940    4924 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:33:19.763007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.762984    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:33:19.763096 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.763013    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:33:19.768198 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.768174    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/1.log"
Oct 31 00:33:19.768333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.768210    4924 generic.go:356] "Generic (PLEG): container finished" podID="041328ee-8f09-4927-85e3-f3cceefe3a53" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a" exitCode=255
Oct 31 00:33:19.774257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.774224    4924 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="c648669e59eaf2d143b3740aca8993af1ac6d588b136b7b7a67ab3e470d6617c" exitCode=0
Oct 31 00:33:19.774257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.774254    4924 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="69574eb4782ce89b3d4ad81c90701d2dbe38b83d562bfe2f1139df7e6768f30d" exitCode=0
Oct 31 00:33:19.774401 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.774263    4924 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="ac402c0a5472325d9f8707e8dfc41f88579dc8108a975424547fe1a3b89064e4" exitCode=0
Oct 31 00:33:19.774401 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.774271    4924 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="16200c0c728bcb654c2baaffb6ed245d450e004695889ea809ada4f43300486a" exitCode=0
Oct 31 00:33:19.774401 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.774279    4924 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="c630a380db9891c410d7511570a1f6f864db18b147d663121f762addbd218a45" exitCode=0
Oct 31 00:33:19.774401 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.774288    4924 generic.go:356] "Generic (PLEG): container finished" podID="9557d837-f593-4d62-ad04-07b2c0b77987" containerID="d833e8c06b1002dc1a89966e507676a06ac371c9a86be6766e6db2b7883ae9c2" exitCode=0
Oct 31 00:33:19.799594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.799556    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:33:19.799973 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.799948    4924 generic.go:356] "Generic (PLEG): container finished" podID="6643e3d6-52ac-4e9d-af24-8a18f92b3748" containerID="001be0ff76a940006bcb72c4e4d8842751ab9bde35c7086efe176e0ba583a514" exitCode=1
Oct 31 00:33:19.814439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.814343    4924 generic.go:356] "Generic (PLEG): container finished" podID="06019dffe47e9f0f78940dad30a48a0f" containerID="4616211d79023616b9c5e2fbe43e9fb1c3efef1948246607f9982b0910606518" exitCode=0
Oct 31 00:33:19.830491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.830467    4924 manager.go:324] Recovery completed
Oct 31 00:33:19.852495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852449    4924 cpu_manager.go:226] "Starting" policy="none"
Oct 31 00:33:19.852495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852482    4924 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 00:33:19.852495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852508    4924 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852653    4924 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852665    4924 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852681    4924 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852687    4924 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852692    4924 policy_none.go:50] "Start"
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852700    4924 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 00:33:19.852790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852709    4924 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 00:33:19.852986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852847    4924 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 00:33:19.852986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852855    4924 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 00:33:19.852986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.852860    4924 policy_none.go:44] "Start"
Oct 31 00:33:19.856499 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:19.856476    4924 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 31 00:33:19.857394 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.857365    4924 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 00:33:19.862184 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.862165    4924 manager.go:341] "Starting Device Plugin manager"
Oct 31 00:33:19.862654 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:19.862200    4924 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 00:33:19.862721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.862657    4924 server.go:87] "Starting device plugin registration server"
Oct 31 00:33:19.862959 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.862947    4924 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 00:33:19.863009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.862959    4924 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 00:33:19.863088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.863065    4924 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 00:33:19.863264 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.863225    4924 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 00:33:19.863372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.863281    4924 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 00:33:19.863372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.863286    4924 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 00:33:19.863475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.863336    4924 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T00:33:19.863257289Z","UUID":"706a52b7-8c63-4293-b605-f5fd47984b14","Handler":null,"Name":"","Endpoint":""}
Oct 31 00:33:19.864649 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:19.864623    4924 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 00:33:19.864994 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.864979    4924 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 00:33:19.865097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.865003    4924 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 00:33:19.963965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.963929    4924 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:33:19.965230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.965211    4924 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:33:19.965321 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.965243    4924 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:33:19.965321 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.965254    4924 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-217.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:33:19.965321 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.965293    4924 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:19.976789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.976767    4924 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:19.976858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:19.976844    4924 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.057627 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.057559    4924 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"]
Oct 31 00:33:20.058345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058316    4924 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.058345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058301    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"06019dffe47e9f0f78940dad30a48a0f","Type":"ContainerStarted","Data":"fe9e1a72e19b458f7820d4df15c673f7483b483174c5836b0da28589c181e02e"}
Oct 31 00:33:20.058483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058356    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"06019dffe47e9f0f78940dad30a48a0f","Type":"ContainerDied","Data":"4616211d79023616b9c5e2fbe43e9fb1c3efef1948246607f9982b0910606518"}
Oct 31 00:33:20.058483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058373    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"06019dffe47e9f0f78940dad30a48a0f","Type":"ContainerStarted","Data":"521c2eacf6ce3fb7dc3979957866467eb8e60fbfb0d01edecb00ef2d6744fd2c"}
Oct 31 00:33:20.058483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058385    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"ece4c01c041ef372ae3784c8037bb134","Type":"ContainerStarted","Data":"b96d92a7e6618c1ee9e0b3bfe82443fcf14f74406f0925301ec044acf875f2f8"}
Oct 31 00:33:20.058483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058396    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal" event={"ID":"ece4c01c041ef372ae3784c8037bb134","Type":"ContainerStarted","Data":"ac4fc205c7f9d46f9d1343cd1994866e7b4708fb935bcb83894139fd2a7ac348"}
Oct 31 00:33:20.058682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.058499    4924 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.072117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.072049    4924 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:33:20.072117 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.072109    4924 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.072280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.072193    4924 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:33:20.072280 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.072225    4924 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.138544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.138508    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ece4c01c041ef372ae3784c8037bb134-config\") pod \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"ece4c01c041ef372ae3784c8037bb134\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.138544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.138551    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.138773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.138580    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.239208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.239164    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.239356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.239219    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.239356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.239242    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ece4c01c041ef372ae3784c8037bb134-config\") pod \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"ece4c01c041ef372ae3784c8037bb134\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.239356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.239182    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.239356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.239283    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ece4c01c041ef372ae3784c8037bb134-config\") pod \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"ece4c01c041ef372ae3784c8037bb134\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.239356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.239348    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06019dffe47e9f0f78940dad30a48a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" (UID: \"06019dffe47e9f0f78940dad30a48a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.706998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.706958    4924 apiserver.go:52] "Watching apiserver"
Oct 31 00:33:20.716372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.716347    4924 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 00:33:20.719110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.718173    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-x2sfq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp","kube-system/kas-connection-checker-685895c7c9-55fsb","openshift-image-registry/image-registry-ff66f9888-h24kh","openshift-insights/insights-runtime-extractor-bcmv5","openshift-ovn-kubernetes/ovnkube-node-nmwrk","kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal","openshift-ingress-canary/ingress-canary-22xfm","openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz","openshift-image-registry/image-registry-f887cdddf-kb24z","openshift-ingress/router-default-64d4b7bcd7-2dvsg","openshift-network-diagnostics/network-check-target-kwth8","openshift-network-console/networking-console-plugin-86c478bb88-v9cnd","openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr","openshift-multus/multus-f4ddz","openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg","openshift-console-operator/console-operator-7cd56b8dbc-f6w7r","openshift-dns/node-resolver-gbgzq","openshift-network-operator/iptables-alerter-d66sh","openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7","kube-system/kas-connection-checker-685895c7c9-llgrf","openshift-multus/network-metrics-daemon-5659t","kube-system/global-pull-secret-syncer-6gbrr","openshift-dns/dns-default-zmn6t","kube-system/konnectivity-agent-hrtzv","openshift-insights/insights-operator-7c4986d449-gk844","openshift-multus/multus-additional-cni-plugins-9tdv7","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x","kube-system/kas-connection-checker-685895c7c9-8dbtk","openshift-image-registry/node-ca-6hznk","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"]
Oct 31 00:33:20.720894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.720874    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.722684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.722662    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.723084 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.723062    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.723441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.723420    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:20.723811 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.723797    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.724251 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.724238    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:20.730560 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.730534    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.731293 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.731278    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.732484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.732436    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 31 00:33:20.732854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.732833    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.732968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.732852    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.732968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.732884    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.733649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.733617    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.733866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.733847    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 31 00:33:20.734012 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.733996    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.736179 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.736142    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hkzgb\""
Oct 31 00:33:20.736885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.736863    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jwzf5\""
Oct 31 00:33:20.738576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738558    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8fmps\""
Oct 31 00:33:20.738669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738598    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wmhfw\""
Oct 31 00:33:20.738711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738673    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-87knn\""
Oct 31 00:33:20.738711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738686    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-v94lw\""
Oct 31 00:33:20.738797 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738727    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-x5t4j\""
Oct 31 00:33:20.738843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738830    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 31 00:33:20.738898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738832    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 00:33:20.738898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738850    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 31 00:33:20.738898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.738863    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:33:20.739019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739006    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 00:33:20.739066 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739045    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.739066 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739052    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 00:33:20.739172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739156    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 31 00:33:20.739260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739248    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739252    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 31 00:33:20.739313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739268    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 31 00:33:20.739313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739292    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739317    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739327    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739292    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739356    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739388    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739400    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739400    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739415    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gfdx5\""
Oct 31 00:33:20.739441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739443    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739449    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739460    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739467    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cvzdq\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739487    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739494    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-qhb2k\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739505    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739521    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739545    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739551    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739558    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739567    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739488    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfl7t\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739571    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739579    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739588    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739596    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739789    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kh74f\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739811    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739842    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739859    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b29sj\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739874    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.739902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739876    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-85kwq\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739925    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5xbxm\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739955    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739980    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739988    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.739990    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740042    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-bw4k9\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740075    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t9skn\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740130    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740191    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740237    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nzws7\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740328    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-z5p2q\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740449    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k5rkn\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740547    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740579    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lfknl\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740590    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740602    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-hwf9w\""
Oct 31 00:33:20.740725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740607    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kxg2f\""
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740775    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zhwq\" (UniqueName: \"kubernetes.io/projected/65327963-1791-42b7-a9a7-4e77b700ed5a-kube-api-access-5zhwq\") pod \"kas-connection-checker-685895c7c9-8dbtk\" (UID: \"65327963-1791-42b7-a9a7-4e77b700ed5a\") " pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740800    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740817    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/041328ee-8f09-4927-85e3-f3cceefe3a53-serving-cert\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740832    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740850    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.740981    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-run\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741102    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-host\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741126    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-tmp\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741168    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-serving-cert\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741195    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-tmp\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741211    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-tmp\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741277    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-tmp\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741319    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c58bcb9-e087-48f9-91a7-cecacedd8242-config\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741334    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-serving-cert\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.741344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741338    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741338    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/041328ee-8f09-4927-85e3-f3cceefe3a53-serving-cert\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741379    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmwt\" (UniqueName: \"kubernetes.io/projected/995d046f-21e7-4152-bdf2-d0954c8ff37f-kube-api-access-pqmwt\") pod \"kas-connection-checker-685895c7c9-55fsb\" (UID: \"995d046f-21e7-4152-bdf2-d0954c8ff37f\") " pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741409    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss65p\" (UniqueName: \"kubernetes.io/projected/3570c076-c980-454d-9b4d-4fbcc88953e0-kube-api-access-ss65p\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741435    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5h6s\" (UniqueName: \"kubernetes.io/projected/44d79171-f637-4834-bfaa-bd57932eb5b4-kube-api-access-j5h6s\") pod \"volume-data-source-validator-58c9f5f46d-9cltd\" (UID: \"44d79171-f637-4834-bfaa-bd57932eb5b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741451    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkv5h\" (UniqueName: \"kubernetes.io/projected/38dad442-b12d-4dc8-9f95-1471ff276929-kube-api-access-fkv5h\") pod \"network-check-source-6f78d59f99-j5zl7\" (UID: \"38dad442-b12d-4dc8-9f95-1471ff276929\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741468    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysconfig\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741485    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj6gj\" (UniqueName: \"kubernetes.io/projected/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-kube-api-access-cj6gj\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741502    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdptw\" (UniqueName: \"kubernetes.io/projected/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-kube-api-access-kdptw\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741547    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-service-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741589    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27m64\" (UniqueName: \"kubernetes.io/projected/2f593360-052d-45fe-8838-7395a5d16b21-kube-api-access-27m64\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741621    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-modprobe-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741649    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741679    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741698    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f593360-052d-45fe-8838-7395a5d16b21-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741712    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-default-certificate\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741731    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741783    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741816    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-conf\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741843    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741854    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f593360-052d-45fe-8838-7395a5d16b21-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741869    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-snapshots\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741954    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741956    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-snapshots\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.741973    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742042    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742079    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.742088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742095    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6hhh\" (UniqueName: \"kubernetes.io/projected/03f1dde4-3764-4c15-821f-54ca5cd3b47f-kube-api-access-f6hhh\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742130    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-default-certificate\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742159    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-lib-modules\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742179    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwdwd\" (UniqueName: \"kubernetes.io/projected/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-kube-api-access-kwdwd\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742179    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742196    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742217    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9b9\" (UniqueName: \"kubernetes.io/projected/041328ee-8f09-4927-85e3-f3cceefe3a53-kube-api-access-bb9b9\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742405    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742432    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742440    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-stats-auth\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742462    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ppkcp\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742474    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dabbb8b-9e70-4c1c-92d7-04d112150395-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742503    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6p62k\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742504    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742545    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wm824\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742551    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-systemd\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742588    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-config\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742618    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-sys\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742640    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742650    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-tuned\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742668    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742678    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4e15376-708a-442b-9439-8bec92eae34c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742687    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-stats-auth\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742709    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmdf6\" (UniqueName: \"kubernetes.io/projected/7c58bcb9-e087-48f9-91a7-cecacedd8242-kube-api-access-rmdf6\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742739    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f593360-052d-45fe-8838-7395a5d16b21-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742789    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742800    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-tuned\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742813    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-trusted-ca\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742822    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742854    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-config\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742888    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742915    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742996    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.742998    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743025    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpnr\" (UniqueName: \"kubernetes.io/projected/1dabbb8b-9e70-4c1c-92d7-04d112150395-kube-api-access-qcpnr\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743058    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743060    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-var-lib-kubelet\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743093    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743115    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743122    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743125    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743176    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-kubernetes\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743205    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743237    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743242    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743354    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.743342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743369    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743372    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743509    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743546    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743551    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c58bcb9-e087-48f9-91a7-cecacedd8242-serving-cert\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743667    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743731    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c58bcb9-e087-48f9-91a7-cecacedd8242-serving-cert\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743755    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743762    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743800    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4smf\" (UniqueName: \"kubernetes.io/projected/ab123256-37b2-4cf7-961b-38c7a0e3be21-kube-api-access-b4smf\") pod \"kas-connection-checker-685895c7c9-llgrf\" (UID: \"ab123256-37b2-4cf7-961b-38c7a0e3be21\") " pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743669    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743822    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.743862    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744317    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744426    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/041328ee-8f09-4927-85e3-f3cceefe3a53-trusted-ca\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744561    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pqlnb\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744582    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744591    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744610    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744639    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744839    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744860    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.744863    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 00:33:20.745091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.745017    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 00:33:20.747452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.747433    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 00:33:20.747915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.747896    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zhwq\" (UniqueName: \"kubernetes.io/projected/65327963-1791-42b7-a9a7-4e77b700ed5a-kube-api-access-5zhwq\") pod \"kas-connection-checker-685895c7c9-8dbtk\" (UID: \"65327963-1791-42b7-a9a7-4e77b700ed5a\") " pod="kube-system/kas-connection-checker-685895c7c9-8dbtk"
Oct 31 00:33:20.748118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.748100    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xglkq\" (UniqueName: \"kubernetes.io/projected/007129ea-79e7-4781-a23c-8491d433c8a9-kube-api-access-xglkq\") pod \"network-check-target-kwth8\" (UID: \"007129ea-79e7-4781-a23c-8491d433c8a9\") " pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:20.748560 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.748537    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmwt\" (UniqueName: \"kubernetes.io/projected/995d046f-21e7-4152-bdf2-d0954c8ff37f-kube-api-access-pqmwt\") pod \"kas-connection-checker-685895c7c9-55fsb\" (UID: \"995d046f-21e7-4152-bdf2-d0954c8ff37f\") " pod="kube-system/kas-connection-checker-685895c7c9-55fsb"
Oct 31 00:33:20.748973 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.748957    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss65p\" (UniqueName: \"kubernetes.io/projected/3570c076-c980-454d-9b4d-4fbcc88953e0-kube-api-access-ss65p\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.751652 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.751629    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c58bcb9-e087-48f9-91a7-cecacedd8242-config\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:20.751806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.751734    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-service-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.751806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.751785    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.752761 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.752732    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dabbb8b-9e70-4c1c-92d7-04d112150395-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.752825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.752790    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4smf\" (UniqueName: \"kubernetes.io/projected/ab123256-37b2-4cf7-961b-38c7a0e3be21-kube-api-access-b4smf\") pod \"kas-connection-checker-685895c7c9-llgrf\" (UID: \"ab123256-37b2-4cf7-961b-38c7a0e3be21\") " pod="kube-system/kas-connection-checker-685895c7c9-llgrf"
Oct 31 00:33:20.752975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.752955    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.753057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.752994    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4e15376-708a-442b-9439-8bec92eae34c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:20.753092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.753064    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f593360-052d-45fe-8838-7395a5d16b21-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:20.753282 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.753268    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkv5h\" (UniqueName: \"kubernetes.io/projected/38dad442-b12d-4dc8-9f95-1471ff276929-kube-api-access-fkv5h\") pod \"network-check-source-6f78d59f99-j5zl7\" (UID: \"38dad442-b12d-4dc8-9f95-1471ff276929\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-j5zl7"
Oct 31 00:33:20.754179 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.754161    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmdf6\" (UniqueName: \"kubernetes.io/projected/7c58bcb9-e087-48f9-91a7-cecacedd8242-kube-api-access-rmdf6\") pod \"service-ca-operator-544d749c8c-p9hdg\" (UID: \"7c58bcb9-e087-48f9-91a7-cecacedd8242\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg"
Oct 31 00:33:20.754467 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.754452    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9b9\" (UniqueName: \"kubernetes.io/projected/041328ee-8f09-4927-85e3-f3cceefe3a53-kube-api-access-bb9b9\") pod \"console-operator-7cd56b8dbc-f6w7r\" (UID: \"041328ee-8f09-4927-85e3-f3cceefe3a53\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:20.755068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755046    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5h6s\" (UniqueName: \"kubernetes.io/projected/44d79171-f637-4834-bfaa-bd57932eb5b4-kube-api-access-j5h6s\") pod \"volume-data-source-validator-58c9f5f46d-9cltd\" (UID: \"44d79171-f637-4834-bfaa-bd57932eb5b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9cltd"
Oct 31 00:33:20.755165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755046    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27m64\" (UniqueName: \"kubernetes.io/projected/2f593360-052d-45fe-8838-7395a5d16b21-kube-api-access-27m64\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-g222x\" (UID: \"2f593360-052d-45fe-8838-7395a5d16b21\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x"
Oct 31 00:33:20.755257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755235    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6hhh\" (UniqueName: \"kubernetes.io/projected/03f1dde4-3764-4c15-821f-54ca5cd3b47f-kube-api-access-f6hhh\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:20.755329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755261    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpnr\" (UniqueName: \"kubernetes.io/projected/1dabbb8b-9e70-4c1c-92d7-04d112150395-kube-api-access-qcpnr\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.755387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755311    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdptw\" (UniqueName: \"kubernetes.io/projected/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-kube-api-access-kdptw\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.755387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755360    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwdwd\" (UniqueName: \"kubernetes.io/projected/ff9c0bff-850c-4b57-b3fe-ea825f6d4767-kube-api-access-kwdwd\") pod \"insights-operator-7c4986d449-gk844\" (UID: \"ff9c0bff-850c-4b57-b3fe-ea825f6d4767\") " pod="openshift-insights/insights-operator-7c4986d449-gk844"
Oct 31 00:33:20.755532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.755515    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.756357 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.756332    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.756535 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.756520    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj6gj\" (UniqueName: \"kubernetes.io/projected/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-kube-api-access-cj6gj\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.819830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.819801    4924 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.819968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.819846    4924 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.826413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.826390    4924 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 00:33:20.832848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.832825    4924 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:33:20.832940 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.832883    4924 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.839060 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.839035    4924 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:33:20.839180 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.839082    4924 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-217.us-east-2.compute.internal"
Oct 31 00:33:20.844737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.844716    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-lib-modules\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.844832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.844756    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-cni-binary-copy\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.844832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.844780    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-binary-copy\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.844832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.844805    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg77m\" (UniqueName: \"kubernetes.io/projected/ef90678c-925d-49bb-acd6-0d2861437916-kube-api-access-sg77m\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.844953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.844922    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-lib-modules\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.844953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.844924    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-systemd-units\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845023    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-binary-copy\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.845075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845035    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-etc-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845071    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-kubelet\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845102    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-systemd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845102    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-cni-binary-copy\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.845228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845147    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t8cq\" (UniqueName: \"kubernetes.io/projected/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-kube-api-access-4t8cq\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:20.845228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845189    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-kubelet-config\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.845284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845239    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-netns\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.845284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845272    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-socket-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.845338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845301    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/542f4323-b31e-4491-9b6e-02a1916798ab-konnectivity-ca\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:33:20.845338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845328    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.845422 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845406    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/542f4323-b31e-4491-9b6e-02a1916798ab-agent-certs\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:33:20.845455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845438    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.845485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845463    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845491    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-script-lib\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845625 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.845593    4924 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:20.845681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845626    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovn-node-metrics-cert\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845650    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.845681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845667    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9fnb\" (UniqueName: \"kubernetes.io/projected/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-kube-api-access-n9fnb\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.845681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845666    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/542f4323-b31e-4491-9b6e-02a1916798ab-agent-certs\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.845697    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.345675292 +0000 UTC m=+1.716459077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845609    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/542f4323-b31e-4491-9b6e-02a1916798ab-konnectivity-ca\") pod \"konnectivity-agent-hrtzv\" (UID: \"542f4323-b31e-4491-9b6e-02a1916798ab\") " pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845722    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845777    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845806    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-conf-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845831    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-system-cni-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845858    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edb0ea49-807e-4139-a8bc-516f3b88b720-host-slash\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845857    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845857    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovn-node-metrics-cert\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.845892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845886    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845900    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-host\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845923    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-serviceca\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845959    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.845975    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-script-lib\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846014    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-host\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846040    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846056    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-bin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846071    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-netd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846079    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-serviceca\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846090    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846094    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-host\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846143    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846190    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-k8s-cni-cncf-io\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.846192    4924 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846211    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-config\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.846252    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.346241815 +0000 UTC m=+1.717025586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : secret "metrics-daemon-secret" not found
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846291    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jx9h\" (UniqueName: \"kubernetes.io/projected/69a61c32-b459-46fa-a720-80637d99c7ae-kube-api-access-8jx9h\") pod \"migrator-66df5d496b-whnqr\" (UID: \"69a61c32-b459-46fa-a720-80637d99c7ae\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:20.846309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846313    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-data-volume\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846329    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-device-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846345    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-hostroot\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846380    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgzq2\" (UniqueName: \"kubernetes.io/projected/9557d837-f593-4d62-ad04-07b2c0b77987-kube-api-access-hgzq2\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846428    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-data-volume\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846442    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-ovnkube-config\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846547    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/edb0ea49-807e-4139-a8bc-516f3b88b720-iptables-alerter-script\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846623    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-bin\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846724    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-modprobe-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846778    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846820    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846847    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-crio-socket\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846854    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-modprobe-d\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.846882    4924 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846886    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846908    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-multus\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846886    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/edb0ea49-807e-4139-a8bc-516f3b88b720-iptables-alerter-script\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.846934    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.34692015 +0000 UTC m=+1.717703931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846950    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846971    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-slash\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.846983    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.846994    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.346979454 +0000 UTC m=+1.717763272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:20.847035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847014    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-registration-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847046    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847078    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-systemd\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.847095    4924 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847098    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9557d837-f593-4d62-ad04-07b2c0b77987-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847101    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-daemon-config\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847128    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-cnibin\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847150    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-ovn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.847178    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.347165129 +0000 UTC m=+1.717948903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847189    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-systemd\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847196    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-log-socket\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847230    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-dbus\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847231    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-daemon-config\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847253    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-sys\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847276    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-kubelet\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847288    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-sys\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847299    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847324    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847341    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-var-lib-kubelet\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847378    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847373    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-var-lib-kubelet\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847399    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-system-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847416    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.847451    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.847461    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847469    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-kubernetes\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.847494    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.347482217 +0000 UTC m=+1.718266023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847517    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-kubernetes\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847517    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-node-log\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847550    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-sys-fs\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847565    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-cnibin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847581    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef90678c-925d-49bb-acd6-0d2861437916-config-volume\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847637    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8mpf\" (UniqueName: \"kubernetes.io/projected/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-api-access-s8mpf\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847658    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-socket-dir-parent\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847673    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-multus-certs\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847674    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c562d34-6b68-4800-a5a6-5c9e204a6050-original-pull-secret\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.847668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847687    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847720    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847741    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847811    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847860    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847916    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef90678c-925d-49bb-acd6-0d2861437916-config-volume\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847918    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.847999    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-cert\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848076    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848095    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848128    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnq72\" (UniqueName: \"kubernetes.io/projected/bb393179-7795-4aa8-bd36-538fe2821808-kube-api-access-rnq72\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848162    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848188    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-cert\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848224    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-run\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848251    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef90678c-925d-49bb-acd6-0d2861437916-tmp-dir\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848297    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-run\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848297    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-netns\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848341    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848362    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848380    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef90678c-925d-49bb-acd6-0d2861437916-tmp-dir\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848392    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848425    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15983cd6-0227-4191-8770-4fd87b6a52b2-tmp-dir\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848445    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck9d4\" (UniqueName: \"kubernetes.io/projected/15983cd6-0227-4191-8770-4fd87b6a52b2-kube-api-access-ck9d4\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848464    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rct8\" (UniqueName: \"kubernetes.io/projected/fce6d408-267e-49f1-ac1e-68176943fbf9-kube-api-access-9rct8\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848486    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848501    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15983cd6-0227-4191-8770-4fd87b6a52b2-hosts-file\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848517    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysconfig\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848514    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15983cd6-0227-4191-8770-4fd87b6a52b2-tmp-dir\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848575    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysconfig\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848689    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-etc-kubernetes\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848719    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-env-overrides\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848763    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwzqt\" (UniqueName: \"kubernetes.io/projected/edb0ea49-807e-4139-a8bc-516f3b88b720-kube-api-access-kwzqt\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848792    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-var-lib-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848819    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848850    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqbn\" (UniqueName: \"kubernetes.io/projected/6643e3d6-52ac-4e9d-af24-8a18f92b3748-kube-api-access-zpqbn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848870    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6643e3d6-52ac-4e9d-af24-8a18f92b3748-env-overrides\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.848898 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.848897    4924 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:20.849873 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.848938    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.348926965 +0000 UTC m=+1.719710765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:20.849873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.848875    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-conf\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.849873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.849015    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dfde58a0-def5-41fb-8f2d-e825a8ab34fe-etc-sysctl-conf\") pod \"tuned-x2sfq\" (UID: \"dfde58a0-def5-41fb-8f2d-e825a8ab34fe\") " pod="openshift-cluster-node-tuning-operator/tuned-x2sfq"
Oct 31 00:33:20.853999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.853983    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t8cq\" (UniqueName: \"kubernetes.io/projected/7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee-kube-api-access-4t8cq\") pod \"ingress-canary-22xfm\" (UID: \"7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee\") " pod="openshift-ingress-canary/ingress-canary-22xfm"
Oct 31 00:33:20.857181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.857162    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9fnb\" (UniqueName: \"kubernetes.io/projected/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-kube-api-access-n9fnb\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.857433 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.857416    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.857547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.857532    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg77m\" (UniqueName: \"kubernetes.io/projected/ef90678c-925d-49bb-acd6-0d2861437916-kube-api-access-sg77m\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.861047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.861031    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rct8\" (UniqueName: \"kubernetes.io/projected/fce6d408-267e-49f1-ac1e-68176943fbf9-kube-api-access-9rct8\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.861634 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.861609    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwzqt\" (UniqueName: \"kubernetes.io/projected/edb0ea49-807e-4139-a8bc-516f3b88b720-kube-api-access-kwzqt\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.861930 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.861910    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnq72\" (UniqueName: \"kubernetes.io/projected/bb393179-7795-4aa8-bd36-538fe2821808-kube-api-access-rnq72\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.861985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.861930    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8mpf\" (UniqueName: \"kubernetes.io/projected/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-api-access-s8mpf\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.861985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.861932    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jx9h\" (UniqueName: \"kubernetes.io/projected/69a61c32-b459-46fa-a720-80637d99c7ae-kube-api-access-8jx9h\") pod \"migrator-66df5d496b-whnqr\" (UID: \"69a61c32-b459-46fa-a720-80637d99c7ae\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-whnqr"
Oct 31 00:33:20.862255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.862240    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck9d4\" (UniqueName: \"kubernetes.io/projected/15983cd6-0227-4191-8770-4fd87b6a52b2-kube-api-access-ck9d4\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.862349 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.862331    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgzq2\" (UniqueName: \"kubernetes.io/projected/9557d837-f593-4d62-ad04-07b2c0b77987-kube-api-access-hgzq2\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.862531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.862510    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.862674 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.862659    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqbn\" (UniqueName: \"kubernetes.io/projected/6643e3d6-52ac-4e9d-af24-8a18f92b3748-kube-api-access-zpqbn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949676    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-systemd-units\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949707    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-etc-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949722    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-kubelet\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949739    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-systemd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949807    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-systemd-units\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949808    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-etc-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949835    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-systemd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949862    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-kubelet\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949882    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-kubelet-config\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949915    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-netns\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.949978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949937    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-socket-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.949982    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950007    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-netns\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950012    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950032    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-kubelet-config\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950035    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-socket-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950055    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-conf-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950061    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950067    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950083    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-system-cni-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950105    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edb0ea49-807e-4139-a8bc-516f3b88b720-host-slash\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950112    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-conf-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950121    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-host\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950139    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950145    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edb0ea49-807e-4139-a8bc-516f3b88b720-host-slash\") pod \"iptables-alerter-d66sh\" (UID: \"edb0ea49-807e-4139-a8bc-516f3b88b720\") " pod="openshift-network-operator/iptables-alerter-d66sh"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950150    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-system-cni-dir\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950161    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-bin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950181    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-netd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950188    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c7b1d82-f9da-4de8-bb8f-f8e76a938aff-host\") pod \"node-ca-6hznk\" (UID: \"4c7b1d82-f9da-4de8-bb8f-f8e76a938aff\") " pod="openshift-image-registry/node-ca-6hznk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950193    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950220    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-netd\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950226    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-bin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950254    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-k8s-cni-cncf-io\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950283    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-device-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950307    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-k8s-cni-cncf-io\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950307    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-hostroot\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950338    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-bin\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950370    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-device-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.950372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950371    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-crio-socket\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950399    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-crio-socket\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950407    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-multus\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950338    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-hostroot\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950431    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-slash\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950370    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-cni-bin\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950452    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-cni-multus\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950454    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-registration-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950481    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-slash\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950494    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-registration-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950508    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-cnibin\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950522    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9557d837-f593-4d62-ad04-07b2c0b77987-cnibin\") pod \"multus-additional-cni-plugins-9tdv7\" (UID: \"9557d837-f593-4d62-ad04-07b2c0b77987\") " pod="openshift-multus/multus-additional-cni-plugins-9tdv7"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950530    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-ovn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950547    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-log-socket\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950585    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-dbus\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950591    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-run-ovn\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950602    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-kubelet\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950619    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950637    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-log-socket\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950649    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-system-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950674    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-var-lib-kubelet\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950676    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-node-log\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950681    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c562d34-6b68-4800-a5a6-5c9e204a6050-dbus\") pod \"global-pull-secret-syncer-6gbrr\" (UID: \"4c562d34-6b68-4800-a5a6-5c9e204a6050\") " pod="kube-system/global-pull-secret-syncer-6gbrr"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950698    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-sys-fs\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.950702    4924 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950720    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-cnibin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950726    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-node-log\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950734    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-system-cni-dir\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950771    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-socket-dir-parent\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.950790    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.450770107 +0000 UTC m=+1.821553888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : configmap "kube-rbac-proxy" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950799    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-cnibin\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950787    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-sys-fs\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950813    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-multus-socket-dir-parent\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950819    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-multus-certs\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950843    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-host-run-multus-certs\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950846    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950870    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950898    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950910    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950923    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950962    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-netns\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.950988    4924 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.950995    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.951007    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951022    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-netns\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.950988    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951022    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.951028    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.451013242 +0000 UTC m=+1.821797010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.950990    4924 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.951110    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.451095651 +0000 UTC m=+1.821879431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:20.951139    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:21.451132949 +0000 UTC m=+1.821916716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951150    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951171    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951186    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15983cd6-0227-4191-8770-4fd87b6a52b2-hosts-file\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951209    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-etc-kubernetes\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951239    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-var-lib-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951242    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb393179-7795-4aa8-bd36-538fe2821808-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7fbp\" (UID: \"bb393179-7795-4aa8-bd36-538fe2821808\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7fbp"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951212    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951263    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15983cd6-0227-4191-8770-4fd87b6a52b2-hosts-file\") pod \"node-resolver-gbgzq\" (UID: \"15983cd6-0227-4191-8770-4fd87b6a52b2\") " pod="openshift-dns/node-resolver-gbgzq"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951240    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fce6d408-267e-49f1-ac1e-68176943fbf9-etc-kubernetes\") pod \"multus-f4ddz\" (UID: \"fce6d408-267e-49f1-ac1e-68176943fbf9\") " pod="openshift-multus/multus-f4ddz"
Oct 31 00:33:20.951441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:20.951277    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6643e3d6-52ac-4e9d-af24-8a18f92b3748-var-lib-openvswitch\") pod \"ovnkube-node-nmwrk\" (UID: \"6643e3d6-52ac-4e9d-af24-8a18f92b3748\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:21.020960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.020915    4924 scope.go:122] "RemoveContainer" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a"
Oct 31 00:33:21.355548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355455    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:21.355548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355511    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:21.355548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355543    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355571    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355617    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355708    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355619    4924 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355784    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355761828 +0000 UTC m=+2.726545610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355800    4924 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355819    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355801052 +0000 UTC m=+2.726584824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : secret "metrics-daemon-secret" not found
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355674    4924 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355840    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355827084 +0000 UTC m=+2.726610870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:21.355875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.355856    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355885    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355873943 +0000 UTC m=+2.726657714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355692    4924 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355923    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355916629 +0000 UTC m=+2.726700396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355922    4924 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355700    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355939    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355948    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355941038 +0000 UTC m=+2.726724831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:21.356318 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.355961    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.355954914 +0000 UTC m=+2.726738680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:21.457031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.456992    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:21.457031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.457037    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.457063    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457145    4924 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457146    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457212    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.457198688 +0000 UTC m=+2.827982468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457217    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.457248    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457264    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.457249631 +0000 UTC m=+2.828033411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:21.457283 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457200    4924 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 00:33:21.457604 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.457337    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls podName:ef90678c-925d-49bb-acd6-0d2861437916 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:22.457312968 +0000 UTC m=+2.828096738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls") pod "dns-default-zmn6t" (UID: "ef90678c-925d-49bb-acd6-0d2861437916") : secret "dns-default-metrics-tls" not found
Oct 31 00:33:21.457810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.457795    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:21.824123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.824095    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:33:21.824516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.824501    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/1.log"
Oct 31 00:33:21.824571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.824532    4924 generic.go:356] "Generic (PLEG): container finished" podID="041328ee-8f09-4927-85e3-f3cceefe3a53" containerID="7e1892b27c776ee2753b566a52585afec615967ece8c898727e1455412492f9f" exitCode=255
Oct 31 00:33:21.824630 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.824606    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" event={"ID":"041328ee-8f09-4927-85e3-f3cceefe3a53","Type":"ContainerDied","Data":"7e1892b27c776ee2753b566a52585afec615967ece8c898727e1455412492f9f"}
Oct 31 00:33:21.824678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.824666    4924 scope.go:122] "RemoveContainer" containerID="e8a41cc4b0cf78b43ac3fedada3a1cf8cfd08aca10dcd15fe75b9847fbf0c42a"
Oct 31 00:33:21.825264 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.825243    4924 scope.go:122] "RemoveContainer" containerID="7e1892b27c776ee2753b566a52585afec615967ece8c898727e1455412492f9f"
Oct 31 00:33:21.825426 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:21.825399    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-f6w7r_openshift-console-operator(041328ee-8f09-4927-85e3-f3cceefe3a53)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53"
Oct 31 00:33:21.996110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:21.996078    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:22.365913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.365879    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:22.366093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.365933    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:22.366093 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366050    4924 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 00:33:22.366163 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366121    4924 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 00:33:22.366163 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.366059    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:22.366163 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366123    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert podName:f4e15376-708a-442b-9439-8bec92eae34c nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366105736 +0000 UTC m=+4.736889527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v9cnd" (UID: "f4e15376-708a-442b-9439-8bec92eae34c") : secret "networking-console-plugin-cert" not found
Oct 31 00:33:22.366261 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366170    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366160003 +0000 UTC m=+4.736943775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : secret "metrics-daemon-secret" not found
Oct 31 00:33:22.366261 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366170    4924 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:22.366261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.366209    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:22.366261 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366253    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366237792 +0000 UTC m=+4.737021562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.366274    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366295    4924 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.366309    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366333    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls podName:791da7b8-431d-4ee0-8bfb-04b22f6d49c8 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366321403 +0000 UTC m=+4.737105182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-zh79b" (UID: "791da7b8-431d-4ee0-8bfb-04b22f6d49c8") : secret "samples-operator-tls" not found
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366369    4924 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.366373    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366415    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366398306 +0000 UTC m=+4.737182079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:22.366435 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366437    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:22.366704 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366452    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:22.366704 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366472    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366461099 +0000 UTC m=+4.737244869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:22.366704 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.366486    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.366480236 +0000 UTC m=+4.737264003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:22.467695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.467658    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:22.467695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.467695    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:22.467950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.467721    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:22.467950 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.467836    4924 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:22.467950 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.467902    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.467888383 +0000 UTC m=+4.838672173 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:22.467950 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.467844    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:22.467950 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.467924    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:22.467950 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.467949    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:24.467942401 +0000 UTC m=+4.838726202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:22.468231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.468200    4924 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 00:33:22.470383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.470352    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef90678c-925d-49bb-acd6-0d2861437916-metrics-tls\") pod \"dns-default-zmn6t\" (UID: \"ef90678c-925d-49bb-acd6-0d2861437916\") " pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:22.532342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.532304    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:22.804895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.804871    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zmn6t"]
Oct 31 00:33:22.807385 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:22.807340    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef90678c_925d_49bb_acd6_0d2861437916.slice/crio-487ef85ec8b992755c3a79d0e0230569ec11f986ab3b106693295a96b49dabd6 WatchSource:0}: Error finding container 487ef85ec8b992755c3a79d0e0230569ec11f986ab3b106693295a96b49dabd6: Status 404 returned error can't find the container with id 487ef85ec8b992755c3a79d0e0230569ec11f986ab3b106693295a96b49dabd6
Oct 31 00:33:22.809492 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.809472    4924 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:33:22.829183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.829156    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:33:22.829577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.829508    4924 scope.go:122] "RemoveContainer" containerID="7e1892b27c776ee2753b566a52585afec615967ece8c898727e1455412492f9f"
Oct 31 00:33:22.829755 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:22.829721    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-f6w7r_openshift-console-operator(041328ee-8f09-4927-85e3-f3cceefe3a53)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53"
Oct 31 00:33:22.830481 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:22.830458    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zmn6t" event={"ID":"ef90678c-925d-49bb-acd6-0d2861437916","Type":"ContainerStarted","Data":"487ef85ec8b992755c3a79d0e0230569ec11f986ab3b106693295a96b49dabd6"}
Oct 31 00:33:23.974631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:23.974598    4924 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hrtzv_542f4323-b31e-4491-9b6e-02a1916798ab/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:24.099289 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.099245    4924 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hrtzv_542f4323-b31e-4491-9b6e-02a1916798ab/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:24.182926 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.182893    4924 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hrtzv_542f4323-b31e-4491-9b6e-02a1916798ab/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:24.354605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.354573    4924 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hrtzv_542f4323-b31e-4491-9b6e-02a1916798ab/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:24.388665 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.388626    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:24.388880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.388670    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:24.388880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.388699    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:24.388880 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.388810    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:24.388880 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.388817    4924 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:24.388880 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.388823    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.388886    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.388865329 +0000 UTC m=+8.759649109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.388920    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.388910201 +0000 UTC m=+8.759693975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.388936    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.388927096 +0000 UTC m=+8.759710865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.388957    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.389019    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.389065    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.389101    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:24.389121 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.389125    4924 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 00:33:24.389440 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.389164    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.389148814 +0000 UTC m=+8.759932584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : secret "router-metrics-certs-default" not found
Oct 31 00:33:24.389440 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.389226    4924 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 00:33:24.389440 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.389259    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs podName:03f1dde4-3764-4c15-821f-54ca5cd3b47f nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.389250648 +0000 UTC m=+8.760034448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs") pod "network-metrics-daemon-5659t" (UID: "03f1dde4-3764-4c15-821f-54ca5cd3b47f") : secret "metrics-daemon-secret" not found
Oct 31 00:33:24.391549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.391528    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/791da7b8-431d-4ee0-8bfb-04b22f6d49c8-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-zh79b\" (UID: \"791da7b8-431d-4ee0-8bfb-04b22f6d49c8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:24.391716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.391687    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4e15376-708a-442b-9439-8bec92eae34c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v9cnd\" (UID: \"f4e15376-708a-442b-9439-8bec92eae34c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:24.490118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.490084    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:24.490118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.490118    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:24.490355 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.490254    4924 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:24.490355 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.490304    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.490292157 +0000 UTC m=+8.861075924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:24.490473 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.490370    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:24.490473 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.490391    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:24.490473 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:24.490448    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:28.490431476 +0000 UTC m=+8.861215266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:24.586095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.586059    4924 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hrtzv_542f4323-b31e-4491-9b6e-02a1916798ab/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:24.624210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.624127    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"
Oct 31 00:33:24.624367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.624240    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"
Oct 31 00:33:24.920969 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.920942    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-v9cnd"]
Oct 31 00:33:24.925534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:24.925502    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b"]
Oct 31 00:33:24.926205 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:24.926179    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e15376_708a_442b_9439_8bec92eae34c.slice/crio-74703433f784ae9919d23b20ad3d5ff0157e6c11a3cf3d94164d98d0b1d78aef WatchSource:0}: Error finding container 74703433f784ae9919d23b20ad3d5ff0157e6c11a3cf3d94164d98d0b1d78aef: Status 404 returned error can't find the container with id 74703433f784ae9919d23b20ad3d5ff0157e6c11a3cf3d94164d98d0b1d78aef
Oct 31 00:33:25.842983 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:25.842945    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b" event={"ID":"791da7b8-431d-4ee0-8bfb-04b22f6d49c8","Type":"ContainerStarted","Data":"87857eccf8472fd52263d13ffbd1d04900e91033654a2497b58a27cbd225bb1d"}
Oct 31 00:33:25.844402 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:25.844371    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd" event={"ID":"f4e15376-708a-442b-9439-8bec92eae34c","Type":"ContainerStarted","Data":"74703433f784ae9919d23b20ad3d5ff0157e6c11a3cf3d94164d98d0b1d78aef"}
Oct 31 00:33:25.847369 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:25.846883    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zmn6t" event={"ID":"ef90678c-925d-49bb-acd6-0d2861437916","Type":"ContainerStarted","Data":"f4c9d9947f9af572b26447c3be1eb2f19016bd4b25c3e7b37e4e9e47fd321673"}
Oct 31 00:33:25.847369 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:25.846930    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zmn6t" event={"ID":"ef90678c-925d-49bb-acd6-0d2861437916","Type":"ContainerStarted","Data":"9068c200bdfa01c55cf6cabc485ad3204b7939970902f56f835de11714d8e54a"}
Oct 31 00:33:25.847369 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:25.847003    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:26.836894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:26.836862    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zmn6t_ef90678c-925d-49bb-acd6-0d2861437916/dns/0.log"
Oct 31 00:33:26.851757 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:26.851721    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-v9cnd" event={"ID":"f4e15376-708a-442b-9439-8bec92eae34c","Type":"ContainerStarted","Data":"719befdfb7eb74e1a727e93b2d3383918e58c5e60fd45e47cccbf46807a9d23a"}
Oct 31 00:33:27.037284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:27.037255    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zmn6t_ef90678c-925d-49bb-acd6-0d2861437916/kube-rbac-proxy/0.log"
Oct 31 00:33:27.636511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:27.636485    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbgzq_15983cd6-0227-4191-8770-4fd87b6a52b2/dns-node-resolver/0.log"
Oct 31 00:33:27.857102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:27.857064    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b" event={"ID":"791da7b8-431d-4ee0-8bfb-04b22f6d49c8","Type":"ContainerStarted","Data":"94797931560b787353fa23cbac5994164e23525d965d82cbf1384e1f8fe766df"}
Oct 31 00:33:27.857536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:27.857127    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-zh79b" event={"ID":"791da7b8-431d-4ee0-8bfb-04b22f6d49c8","Type":"ContainerStarted","Data":"c7a36df141220db50a3a3c740baf23ed7dd9ca15dbc945c2f66a0d2e2bde3bd9"}
Oct 31 00:33:28.374228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.374185    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:28.426496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.426459    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:28.426496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.426498    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:28.426711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.426528    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") pod \"image-registry-ff66f9888-h24kh\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") " pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:28.426711 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.426622    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle podName:3570c076-c980-454d-9b4d-4fbcc88953e0 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:36.426604531 +0000 UTC m=+16.797388298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle") pod "router-default-64d4b7bcd7-2dvsg" (UID: "3570c076-c980-454d-9b4d-4fbcc88953e0") : configmap references non-existent config key: service-ca.crt
Oct 31 00:33:28.426711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.426652    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:28.426711 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.426673    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:28.426711 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.426688    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ff66f9888-h24kh: secret "image-registry-tls" not found
Oct 31 00:33:28.426711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.426697    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:28.426951 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.426734    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls podName:b395282f-6fab-4ee4-b581-c5194700dc6b nodeName:}" failed. No retries permitted until 2026-10-31 00:33:36.426717151 +0000 UTC m=+16.797500935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls") pod "image-registry-ff66f9888-h24kh" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b") : secret "image-registry-tls" not found
Oct 31 00:33:28.426951 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.426627    4924 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:28.426951 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.426791    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls podName:1dabbb8b-9e70-4c1c-92d7-04d112150395 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:36.42678069 +0000 UTC m=+16.797564457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-nsxqz" (UID: "1dabbb8b-9e70-4c1c-92d7-04d112150395") : secret "cluster-monitoring-operator-tls" not found
Oct 31 00:33:28.429091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.429070    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03f1dde4-3764-4c15-821f-54ca5cd3b47f-metrics-certs\") pod \"network-metrics-daemon-5659t\" (UID: \"03f1dde4-3764-4c15-821f-54ca5cd3b47f\") " pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:28.429091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.429091    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3570c076-c980-454d-9b4d-4fbcc88953e0-metrics-certs\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:28.435301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.435282    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6hznk_4c7b1d82-f9da-4de8-bb8f-f8e76a938aff/node-ca/0.log"
Oct 31 00:33:28.524913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.524882    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5659t"
Oct 31 00:33:28.527230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.527209    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:28.527289 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.527249    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:28.527363 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.527346    4924 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 00:33:28.527405 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.527364    4924 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f887cdddf-kb24z: secret "image-registry-tls" not found
Oct 31 00:33:28.527451 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.527408    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls podName:9db3fd71-290b-4156-afe5-af15a922b238 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:36.527393799 +0000 UTC m=+16.898177589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls") pod "image-registry-f887cdddf-kb24z" (UID: "9db3fd71-290b-4156-afe5-af15a922b238") : secret "image-registry-tls" not found
Oct 31 00:33:28.527451 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.527415    4924 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:28.527522 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:28.527460    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls podName:a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc nodeName:}" failed. No retries permitted until 2026-10-31 00:33:36.527447295 +0000 UTC m=+16.898231099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bcmv5" (UID: "a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc") : secret "insights-runtime-extractor-tls" not found
Oct 31 00:33:28.798737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.798706    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5659t"]
Oct 31 00:33:28.801970 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:28.801941    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f1dde4_3764_4c15_821f_54ca5cd3b47f.slice/crio-4b8f147012d7e85085000d1b143b460000b5691228d00246f1bcefc4a2766d3b WatchSource:0}: Error finding container 4b8f147012d7e85085000d1b143b460000b5691228d00246f1bcefc4a2766d3b: Status 404 returned error can't find the container with id 4b8f147012d7e85085000d1b143b460000b5691228d00246f1bcefc4a2766d3b
Oct 31 00:33:28.840530 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.840503    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:28.861659 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:28.861629    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5659t" event={"ID":"03f1dde4-3764-4c15-821f-54ca5cd3b47f","Type":"ContainerStarted","Data":"4b8f147012d7e85085000d1b143b460000b5691228d00246f1bcefc4a2766d3b"}
Oct 31 00:33:29.040863 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:29.040799    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22xfm_7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/serve-healthcheck-canary/0.log"
Oct 31 00:33:29.421807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:29.421106    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:29.421807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:29.421315    4924 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:33:29.638087 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:29.638061    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-whnqr_69a61c32-b459-46fa-a720-80637d99c7ae/migrator/0.log"
Oct 31 00:33:29.836892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:29.836860    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-g222x_2f593360-052d-45fe-8838-7395a5d16b21/kube-storage-version-migrator-operator/0.log"
Oct 31 00:33:30.109535 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:30.109509    4924 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:30.109914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:30.109900    4924 scope.go:122] "RemoveContainer" containerID="7e1892b27c776ee2753b566a52585afec615967ece8c898727e1455412492f9f"
Oct 31 00:33:30.110125 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:30.110105    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-f6w7r_openshift-console-operator(041328ee-8f09-4927-85e3-f3cceefe3a53)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53"
Oct 31 00:33:30.873538 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:30.873502    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5659t" event={"ID":"03f1dde4-3764-4c15-821f-54ca5cd3b47f","Type":"ContainerStarted","Data":"1b356c69f3643eb284422e378580f40a57c04c284f6883f80ed74a58bba50acf"}
Oct 31 00:33:30.873721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:30.873557    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5659t" event={"ID":"03f1dde4-3764-4c15-821f-54ca5cd3b47f","Type":"ContainerStarted","Data":"5d532f3a0770434a2d19ce704ffb14cbad73add2583ce0a35ed9bbdda06daa56"}
Oct 31 00:33:31.757438 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:31.757398    4924 scope.go:122] "RemoveContainer" containerID="7e1892b27c776ee2753b566a52585afec615967ece8c898727e1455412492f9f"
Oct 31 00:33:31.878901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:31.878877    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:33:31.879059 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:31.878992    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" event={"ID":"041328ee-8f09-4927-85e3-f3cceefe3a53","Type":"ContainerStarted","Data":"9d4343b6c94d90b5ae11a5031ef5531bdd145c270ed11aabaae3bf46fc22fe6b"}
Oct 31 00:33:31.879627 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:31.879584    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:32.089913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:32.089883    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r"
Oct 31 00:33:34.403599 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.403564    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ff66f9888-h24kh"]
Oct 31 00:33:34.404017 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:34.403785    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-ff66f9888-h24kh" podUID="b395282f-6fab-4ee4-b581-c5194700dc6b"
Oct 31 00:33:34.889096 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.889069    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:34.895613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.895590    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:34.987391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987351    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987397    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987433    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987490    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987518    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987547    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987579    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") pod \"b395282f-6fab-4ee4-b581-c5194700dc6b\" (UID: \"b395282f-6fab-4ee4-b581-c5194700dc6b\") "
Oct 31 00:33:34.987935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987662    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:33:34.987935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987919    4924 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b395282f-6fab-4ee4-b581-c5194700dc6b-ca-trust-extracted\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:34.988045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.987985    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:33:34.988128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.988106    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:33:34.989944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.989915    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:33:34.990048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.989971    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "kube-api-access-thtgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:33:34.990204 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.990176    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:33:34.990244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:34.990220    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration" pod "b395282f-6fab-4ee4-b581-c5194700dc6b" (UID: "b395282f-6fab-4ee4-b581-c5194700dc6b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:33:35.088830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.088790    4924 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-certificates\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:35.088830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.088825    4924 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-image-registry-private-configuration\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:35.088830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.088836    4924 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b395282f-6fab-4ee4-b581-c5194700dc6b-trusted-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:35.089058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.088845    4924 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b395282f-6fab-4ee4-b581-c5194700dc6b-installation-pull-secrets\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:35.089058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.088855    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thtgj\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-kube-api-access-thtgj\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:35.089058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.088865    4924 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-bound-sa-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:35.854605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.854571    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zmn6t"
Oct 31 00:33:35.893534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.893503    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66f9888-h24kh"
Oct 31 00:33:35.939207 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.939173    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ff66f9888-h24kh"]
Oct 31 00:33:35.949296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:35.949265    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-ff66f9888-h24kh"]
Oct 31 00:33:36.098255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.098170    4924 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b395282f-6fab-4ee4-b581-c5194700dc6b-registry-tls\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:36.500715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.500620    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:36.500715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.500668    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:36.501315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.501289    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3570c076-c980-454d-9b4d-4fbcc88953e0-service-ca-bundle\") pod \"router-default-64d4b7bcd7-2dvsg\" (UID: \"3570c076-c980-454d-9b4d-4fbcc88953e0\") " pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:36.503079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.503050    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dabbb8b-9e70-4c1c-92d7-04d112150395-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-nsxqz\" (UID: \"1dabbb8b-9e70-4c1c-92d7-04d112150395\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:36.601985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.601948    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:36.601985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.601991    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:36.604447 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.604412    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bcmv5\" (UID: \"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc\") " pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:36.604564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.604467    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"image-registry-f887cdddf-kb24z\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") " pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:36.621429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.621394    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:36.623498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.623477    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"
Oct 31 00:33:36.631650 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.631622    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:36.633786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.633762    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bcmv5"
Oct 31 00:33:36.920987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.920954    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d4b7bcd7-2dvsg"]
Oct 31 00:33:36.923935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.923909    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz"]
Oct 31 00:33:36.924477 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:36.924448    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3570c076_c980_454d_9b4d_4fbcc88953e0.slice/crio-ab791f3802f4422485be67ad17e6125687061fd4fbcfb54c4204fb40916f8f6c WatchSource:0}: Error finding container ab791f3802f4422485be67ad17e6125687061fd4fbcfb54c4204fb40916f8f6c: Status 404 returned error can't find the container with id ab791f3802f4422485be67ad17e6125687061fd4fbcfb54c4204fb40916f8f6c
Oct 31 00:33:36.927257 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:36.927236    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dabbb8b_9e70_4c1c_92d7_04d112150395.slice/crio-313a5b30a5c4a6d3299dbc620963cbdb5687a916fe3f62ddfad5a854afbf323a WatchSource:0}: Error finding container 313a5b30a5c4a6d3299dbc620963cbdb5687a916fe3f62ddfad5a854afbf323a: Status 404 returned error can't find the container with id 313a5b30a5c4a6d3299dbc620963cbdb5687a916fe3f62ddfad5a854afbf323a
Oct 31 00:33:36.931108 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.931083    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bcmv5"]
Oct 31 00:33:36.935096 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:36.935074    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56b54ef_4dcb_4a3a_9043_1a9a1c31f9bc.slice/crio-d99a48726b2bbc197451a2c68b4e6848cd8c0041a513fa7fd7f833a808a73af2 WatchSource:0}: Error finding container d99a48726b2bbc197451a2c68b4e6848cd8c0041a513fa7fd7f833a808a73af2: Status 404 returned error can't find the container with id d99a48726b2bbc197451a2c68b4e6848cd8c0041a513fa7fd7f833a808a73af2
Oct 31 00:33:36.941314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:36.939611    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f887cdddf-kb24z"]
Oct 31 00:33:36.965936 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:36.965907    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db3fd71_290b_4156_afe5_af15a922b238.slice/crio-7f0d17ab5d75e1287948c90693db373ad2e954360fd3b3a2a88ec5ec25ce036f WatchSource:0}: Error finding container 7f0d17ab5d75e1287948c90693db373ad2e954360fd3b3a2a88ec5ec25ce036f: Status 404 returned error can't find the container with id 7f0d17ab5d75e1287948c90693db373ad2e954360fd3b3a2a88ec5ec25ce036f
Oct 31 00:33:37.763484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.763449    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b395282f-6fab-4ee4-b581-c5194700dc6b" path="/var/lib/kubelet/pods/b395282f-6fab-4ee4-b581-c5194700dc6b/volumes"
Oct 31 00:33:37.903211 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.903178    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f887cdddf-kb24z" event={"ID":"9db3fd71-290b-4156-afe5-af15a922b238","Type":"ContainerStarted","Data":"a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f"}
Oct 31 00:33:37.903355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.903235    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f887cdddf-kb24z" event={"ID":"9db3fd71-290b-4156-afe5-af15a922b238","Type":"ContainerStarted","Data":"7f0d17ab5d75e1287948c90693db373ad2e954360fd3b3a2a88ec5ec25ce036f"}
Oct 31 00:33:37.903355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.903286    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:37.905023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.905000    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg" event={"ID":"3570c076-c980-454d-9b4d-4fbcc88953e0","Type":"ContainerStarted","Data":"04e7abb9babf1fef1dd8edc4ef86250641b85782727bc7135116809ff2956331"}
Oct 31 00:33:37.905152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.905038    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg" event={"ID":"3570c076-c980-454d-9b4d-4fbcc88953e0","Type":"ContainerStarted","Data":"ab791f3802f4422485be67ad17e6125687061fd4fbcfb54c4204fb40916f8f6c"}
Oct 31 00:33:37.906956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.906907    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz" event={"ID":"1dabbb8b-9e70-4c1c-92d7-04d112150395","Type":"ContainerStarted","Data":"313a5b30a5c4a6d3299dbc620963cbdb5687a916fe3f62ddfad5a854afbf323a"}
Oct 31 00:33:37.909078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.909058    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bcmv5" event={"ID":"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc","Type":"ContainerStarted","Data":"ce6f44933ec7ee3f798dc31d049d5e996db0a86f48db498c37991a2d855d6b12"}
Oct 31 00:33:37.909168 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:37.909104    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bcmv5" event={"ID":"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc","Type":"ContainerStarted","Data":"d99a48726b2bbc197451a2c68b4e6848cd8c0041a513fa7fd7f833a808a73af2"}
Oct 31 00:33:38.621609 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:38.621565    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:38.624290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:38.624264    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:38.920693 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:38.920601    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bcmv5" event={"ID":"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc","Type":"ContainerStarted","Data":"58b4a181f544789ea59ed9e7904e39c49f8fb98f76f2179452483b6835349c86"}
Oct 31 00:33:38.923314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:38.923279    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-nsxqz" event={"ID":"1dabbb8b-9e70-4c1c-92d7-04d112150395","Type":"ContainerStarted","Data":"f73ec2da3b2ad922c10fef062e84f7de1538c04e17152634901f314d0c8bfd47"}
Oct 31 00:33:38.923738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:38.923714    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:38.925127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:38.925107    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64d4b7bcd7-2dvsg"
Oct 31 00:33:39.041941 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:39.041911    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hrtzv"
Oct 31 00:33:39.167170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:39.167136    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmwrk"
Oct 31 00:33:39.928888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:39.928851    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bcmv5" event={"ID":"a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc","Type":"ContainerStarted","Data":"e2eeb981dcd9e4e4141fa75ced301a55878bcabbebe329486cc0afc3e33bd127"}
Oct 31 00:33:48.377442 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.377412    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kwth8"
Oct 31 00:33:48.689935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.689848    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nwcch"]
Oct 31 00:33:48.723540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.723508    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.779040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.778980    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 00:33:48.779040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.778998    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kgtx2\""
Oct 31 00:33:48.779240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.779062    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 00:33:48.797976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.797946    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 00:33:48.797976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.797953    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 00:33:48.898533 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898493    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-wtmp\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898533 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898536    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898555    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898608    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-root\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898649    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f939d23-82c9-43f0-be5f-4799bba43a7c-metrics-client-ca\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898694    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-textfile\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898770    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-sys\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898954 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898802    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-tls\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.898954 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.898834    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q665b\" (UniqueName: \"kubernetes.io/projected/8f939d23-82c9-43f0-be5f-4799bba43a7c-kube-api-access-q665b\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.999778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999681    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-tls\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:48.999778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999728    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q665b\" (UniqueName: \"kubernetes.io/projected/8f939d23-82c9-43f0-be5f-4799bba43a7c-kube-api-access-q665b\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999786    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-wtmp\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999807    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999828    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999859    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-root\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999896    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f939d23-82c9-43f0-be5f-4799bba43a7c-metrics-client-ca\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999919    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-textfile\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:48.999951    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-sys\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000368 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.000026    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-sys\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000368 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.000278    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-wtmp\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000368 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.000352    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-textfile\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000524 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.000374    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f939d23-82c9-43f0-be5f-4799bba43a7c-root\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000524 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.000502    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.000682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.000606    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f939d23-82c9-43f0-be5f-4799bba43a7c-metrics-client-ca\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.002519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.002492    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-tls\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.002795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.002775    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f939d23-82c9-43f0-be5f-4799bba43a7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.011305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.011272    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q665b\" (UniqueName: \"kubernetes.io/projected/8f939d23-82c9-43f0-be5f-4799bba43a7c-kube-api-access-q665b\") pod \"node-exporter-nwcch\" (UID: \"8f939d23-82c9-43f0-be5f-4799bba43a7c\") " pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.073210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.073166    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nwcch"
Oct 31 00:33:49.119657 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:33:49.119626    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f939d23_82c9_43f0_be5f_4799bba43a7c.slice/crio-060c6e900bd4123ed89b4a824340a22eeeb48497023ef8bc3965053831589172 WatchSource:0}: Error finding container 060c6e900bd4123ed89b4a824340a22eeeb48497023ef8bc3965053831589172: Status 404 returned error can't find the container with id 060c6e900bd4123ed89b4a824340a22eeeb48497023ef8bc3965053831589172
Oct 31 00:33:49.966395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:49.966347    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwcch" event={"ID":"8f939d23-82c9-43f0-be5f-4799bba43a7c","Type":"ContainerStarted","Data":"060c6e900bd4123ed89b4a824340a22eeeb48497023ef8bc3965053831589172"}
Oct 31 00:33:50.971460 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:50.971359    4924 generic.go:356] "Generic (PLEG): container finished" podID="8f939d23-82c9-43f0-be5f-4799bba43a7c" containerID="fca7dfeac24f31470bbc1fd66162c27a5ff4de0217e1b68d09c95810ba043f49" exitCode=0
Oct 31 00:33:50.971460 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:50.971440    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwcch" event={"ID":"8f939d23-82c9-43f0-be5f-4799bba43a7c","Type":"ContainerDied","Data":"fca7dfeac24f31470bbc1fd66162c27a5ff4de0217e1b68d09c95810ba043f49"}
Oct 31 00:33:51.977802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:51.977763    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwcch" event={"ID":"8f939d23-82c9-43f0-be5f-4799bba43a7c","Type":"ContainerStarted","Data":"f0acb219b6b2fe2faca68aa44015e5e796d7772139b9f83c7d1f8a585c49fa8b"}
Oct 31 00:33:51.978172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:51.977823    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwcch" event={"ID":"8f939d23-82c9-43f0-be5f-4799bba43a7c","Type":"ContainerStarted","Data":"af6c2c498f6374f94536e19ab4443a0c93393e5424baf2fbefc0806c144c27f2"}
Oct 31 00:33:52.001301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:52.001251    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nwcch" podStartSLOduration=2.440041278 podStartE2EDuration="4.001238981s" totalImagesPullingTime="1.481360458s" totalInitContainerRuntime="79.837245ms" isStatefulPod=true podCreationTimestamp="2026-10-31 00:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:52.000992382 +0000 UTC m=+32.371776171" watchObservedRunningTime="2026-10-31 00:33:52.001238981 +0000 UTC m=+32.372022761"
Oct 31 00:33:56.263227 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:56.263187    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f887cdddf-kb24z"]
Oct 31 00:33:56.267965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:33:56.267933    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:33:57.852454 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:33:57.852418    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:34:21.283856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.283817    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f887cdddf-kb24z" podUID="9db3fd71-290b-4156-afe5-af15a922b238" containerName="registry" containerID="cri-o://a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f" gracePeriod=30
Oct 31 00:34:21.535045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.534985    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:34:21.652414 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652372    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652440    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652463    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652491    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652511    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652537    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652582    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652612    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") pod \"9db3fd71-290b-4156-afe5-af15a922b238\" (UID: \"9db3fd71-290b-4156-afe5-af15a922b238\") "
Oct 31 00:34:21.652992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.652947    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:34:21.653232 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.653201    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:34:21.655348 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.655306    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:34:21.655348 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.655324    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "kube-api-access-t2t9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:34:21.655348 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.655307    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:34:21.655518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.655418    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:34:21.655518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.655444    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:34:21.659559 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.659524    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted" pod "9db3fd71-290b-4156-afe5-af15a922b238" (UID: "9db3fd71-290b-4156-afe5-af15a922b238"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:34:21.753328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753296    4924 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9db3fd71-290b-4156-afe5-af15a922b238-ca-trust-extracted\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753325    4924 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-registry-certificates\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753338    4924 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-bound-sa-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753348    4924 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9db3fd71-290b-4156-afe5-af15a922b238-trusted-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753359    4924 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-registry-tls\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753369    4924 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-image-registry-private-configuration\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753378    4924 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9db3fd71-290b-4156-afe5-af15a922b238-installation-pull-secrets\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:21.753534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:21.753388    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2t9g\" (UniqueName: \"kubernetes.io/projected/9db3fd71-290b-4156-afe5-af15a922b238-kube-api-access-t2t9g\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:34:22.096481 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.096449    4924 generic.go:356] "Generic (PLEG): container finished" podID="9db3fd71-290b-4156-afe5-af15a922b238" containerID="a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f" exitCode=0
Oct 31 00:34:22.096654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.096497    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f887cdddf-kb24z"
Oct 31 00:34:22.096654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.096518    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f887cdddf-kb24z" event={"ID":"9db3fd71-290b-4156-afe5-af15a922b238","Type":"ContainerDied","Data":"a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f"}
Oct 31 00:34:22.096654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.096560    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f887cdddf-kb24z" event={"ID":"9db3fd71-290b-4156-afe5-af15a922b238","Type":"ContainerDied","Data":"7f0d17ab5d75e1287948c90693db373ad2e954360fd3b3a2a88ec5ec25ce036f"}
Oct 31 00:34:22.096654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.096579    4924 scope.go:122] "RemoveContainer" containerID="a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f"
Oct 31 00:34:22.104893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.104861    4924 scope.go:122] "RemoveContainer" containerID="a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f"
Oct 31 00:34:22.105167 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:34:22.105147    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f\": container with ID starting with a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f not found: ID does not exist" containerID="a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f"
Oct 31 00:34:22.105219 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.105177    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f"} err="failed to get container status \"a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f\": rpc error: code = NotFound desc = could not find container \"a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f\": container with ID starting with a4158cb1c7c20eb520579559f32c9a6b0941bfe09f9c4c27669bb1cdcf7eab5f not found: ID does not exist"
Oct 31 00:34:22.112212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.112181    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f887cdddf-kb24z"]
Oct 31 00:34:22.121351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:22.121321    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f887cdddf-kb24z"]
Oct 31 00:34:23.761710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:23.761670    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9db3fd71-290b-4156-afe5-af15a922b238" path="/var/lib/kubelet/pods/9db3fd71-290b-4156-afe5-af15a922b238/volumes"
Oct 31 00:34:26.114630 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:26.114592    4924 generic.go:356] "Generic (PLEG): container finished" podID="2f593360-052d-45fe-8838-7395a5d16b21" containerID="fc291e6a13a0eceb15870a2f414df93b9db56e41142413579a26049cc56b2330" exitCode=0
Oct 31 00:34:26.115053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:26.114664    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x" event={"ID":"2f593360-052d-45fe-8838-7395a5d16b21","Type":"ContainerDied","Data":"fc291e6a13a0eceb15870a2f414df93b9db56e41142413579a26049cc56b2330"}
Oct 31 00:34:26.115053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:26.115004    4924 scope.go:122] "RemoveContainer" containerID="fc291e6a13a0eceb15870a2f414df93b9db56e41142413579a26049cc56b2330"
Oct 31 00:34:27.120956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:27.120920    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-g222x" event={"ID":"2f593360-052d-45fe-8838-7395a5d16b21","Type":"ContainerStarted","Data":"3f68b6160b2e15f3eca8a16e28c1e5da7f92fb754c3345bd76232c12b3fd2148"}
Oct 31 00:34:27.851773 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:34:27.851722    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:34:46.194317 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:46.194280    4924 generic.go:356] "Generic (PLEG): container finished" podID="7c58bcb9-e087-48f9-91a7-cecacedd8242" containerID="a1644eef1f75f6a9d19f9e08ed03654af928331122ef5a9e5feff7e67ee3e5f4" exitCode=0
Oct 31 00:34:46.194765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:46.194357    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg" event={"ID":"7c58bcb9-e087-48f9-91a7-cecacedd8242","Type":"ContainerDied","Data":"a1644eef1f75f6a9d19f9e08ed03654af928331122ef5a9e5feff7e67ee3e5f4"}
Oct 31 00:34:46.194765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:46.194710    4924 scope.go:122] "RemoveContainer" containerID="a1644eef1f75f6a9d19f9e08ed03654af928331122ef5a9e5feff7e67ee3e5f4"
Oct 31 00:34:46.195977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:46.195949    4924 generic.go:356] "Generic (PLEG): container finished" podID="ff9c0bff-850c-4b57-b3fe-ea825f6d4767" containerID="cdcc19f0958c4c462de6ef56540aa6fbf63e2bde07d8a068db16ba9d9550a20a" exitCode=0
Oct 31 00:34:46.196090 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:46.196023    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-gk844" event={"ID":"ff9c0bff-850c-4b57-b3fe-ea825f6d4767","Type":"ContainerDied","Data":"cdcc19f0958c4c462de6ef56540aa6fbf63e2bde07d8a068db16ba9d9550a20a"}
Oct 31 00:34:46.196360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:46.196345    4924 scope.go:122] "RemoveContainer" containerID="cdcc19f0958c4c462de6ef56540aa6fbf63e2bde07d8a068db16ba9d9550a20a"
Oct 31 00:34:47.202000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:47.201961    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-p9hdg" event={"ID":"7c58bcb9-e087-48f9-91a7-cecacedd8242","Type":"ContainerStarted","Data":"54803e1fe2aa51e75ee23c81bf96f5314d6d2142a4657572cd51fabfe40cf110"}
Oct 31 00:34:47.203909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:34:47.203885    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-gk844" event={"ID":"ff9c0bff-850c-4b57-b3fe-ea825f6d4767","Type":"ContainerStarted","Data":"8fc63e8d495fb01c4d738adbf360f6f8e19d44036c1d2f7943016787cb1f5c0a"}
Oct 31 00:34:57.852270 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:34:57.852222    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:35:27.852446 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:35:27.852405    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:35:57.851105 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:35:57.851067    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:36:27.852965 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:36:27.852924    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:36:57.851737 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:36:57.851696    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:37:22.286922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.286882    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"]
Oct 31 00:37:22.287377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.287213    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9db3fd71-290b-4156-afe5-af15a922b238" containerName="registry"
Oct 31 00:37:22.287377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.287226    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9db3fd71-290b-4156-afe5-af15a922b238" containerName="registry"
Oct 31 00:37:22.287377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.287299    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9db3fd71-290b-4156-afe5-af15a922b238" containerName="registry"
Oct 31 00:37:22.290368 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.290340    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.300280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.300252    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 31 00:37:22.300391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.300275    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-dcckl\""
Oct 31 00:37:22.300391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.300295    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 31 00:37:22.300545 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.300529    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"]
Oct 31 00:37:22.325868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.325827    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.325868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.325873    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.326130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.326044    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnzdv\" (UniqueName: \"kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.422959 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.422919    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"]
Oct 31 00:37:22.426418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.426396    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.426626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.426603    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.426695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.426652    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.426783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.426769    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzdv\" (UniqueName: \"kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.427069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.427047    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.427069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.427061    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.440146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.440120    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-6lnvq\""
Oct 31 00:37:22.454315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.454286    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"]
Oct 31 00:37:22.456508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.456483    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnzdv\" (UniqueName: \"kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.527932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.527896    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.528105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.527948    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.528105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.528068    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnv4m\" (UniqueName: \"kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.606786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.606697    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:37:22.628939 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.628910    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.629074 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.628968    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4m\" (UniqueName: \"kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.629074 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.629057    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.629390 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.629353    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.629466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.629403    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.641926 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.641895    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnv4m\" (UniqueName: \"kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.742417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.742376    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:22.884765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:22.884677    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"]
Oct 31 00:37:22.887730 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:22.887691    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d8cf0e_9834_41ea_b84f_fff476a6e5dd.slice/crio-6ec46af0e3a979e81ee81605c7fd69bc0f59bcc233f0dc87eb6d1fccc7056061 WatchSource:0}: Error finding container 6ec46af0e3a979e81ee81605c7fd69bc0f59bcc233f0dc87eb6d1fccc7056061: Status 404 returned error can't find the container with id 6ec46af0e3a979e81ee81605c7fd69bc0f59bcc233f0dc87eb6d1fccc7056061
Oct 31 00:37:23.014561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:23.014531    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"]
Oct 31 00:37:23.017824 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:23.017793    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26544eba_faec_40fb_9c51_e9017cfa111d.slice/crio-ca631b63d96adf4ef81ab2faff12c59bfbdd6cc37ff0d1f3f6d20303e42ad93a WatchSource:0}: Error finding container ca631b63d96adf4ef81ab2faff12c59bfbdd6cc37ff0d1f3f6d20303e42ad93a: Status 404 returned error can't find the container with id ca631b63d96adf4ef81ab2faff12c59bfbdd6cc37ff0d1f3f6d20303e42ad93a
Oct 31 00:37:23.765150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:23.765113    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8" event={"ID":"26544eba-faec-40fb-9c51-e9017cfa111d","Type":"ContainerStarted","Data":"ca631b63d96adf4ef81ab2faff12c59bfbdd6cc37ff0d1f3f6d20303e42ad93a"}
Oct 31 00:37:23.766622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:23.766586    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms" event={"ID":"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd","Type":"ContainerStarted","Data":"6ec46af0e3a979e81ee81605c7fd69bc0f59bcc233f0dc87eb6d1fccc7056061"}
Oct 31 00:37:26.475481 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.475435    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"]
Oct 31 00:37:26.479491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.479461    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.483630 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.483604    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-kmbxg\""
Oct 31 00:37:26.488512 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.488484    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"]
Oct 31 00:37:26.564848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.564807    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.565178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.564885    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.565178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.564991    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvqr\" (UniqueName: \"kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.665488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.665449    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.665649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.665523    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.665649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.665570    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvqr\" (UniqueName: \"kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.666008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.665952    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.666259 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.666236    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.674384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.674350    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvqr\" (UniqueName: \"kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:26.800073 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:26.800032    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:37:27.086230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.086141    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"]
Oct 31 00:37:27.090408 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:27.090375    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e2a645_1820_4ab4_8d23_05c6bba2197a.slice/crio-d95052cb365bc35255a7c540724e8738b6f6de0de8bebdf06a0090ecee359b53 WatchSource:0}: Error finding container d95052cb365bc35255a7c540724e8738b6f6de0de8bebdf06a0090ecee359b53: Status 404 returned error can't find the container with id d95052cb365bc35255a7c540724e8738b6f6de0de8bebdf06a0090ecee359b53
Oct 31 00:37:27.158548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.158508    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"]
Oct 31 00:37:27.162682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.162657    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.171063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.171024    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-xnn9b\""
Oct 31 00:37:27.173778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.173651    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"]
Oct 31 00:37:27.256502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.256464    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"]
Oct 31 00:37:27.260524 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.260495    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.269103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.269067    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"]
Oct 31 00:37:27.269508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.269437    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-sq4p8\""
Oct 31 00:37:27.272224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.272195    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.272357 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.272254    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5zts\" (UniqueName: \"kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.272357 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.272340    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.359741 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.359657    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"]
Oct 31 00:37:27.363809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.363780    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.368497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.368466    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-khqn5\""
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373115    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chjnw\" (UniqueName: \"kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373177    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373241    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373306    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373330    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373354    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5zts\" (UniqueName: \"kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.373819    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"]
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.374029    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.374109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.374069    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.390620 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.390591    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5zts\" (UniqueName: \"kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.456517 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.456473    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"]
Oct 31 00:37:27.460324 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.460288    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.466711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.466677    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-xgc9g\""
Oct 31 00:37:27.468040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.468009    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"]
Oct 31 00:37:27.474398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.474367    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.474539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.474423    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chjnw\" (UniqueName: \"kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.474539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.474458    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.474539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.474513    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drlr7\" (UniqueName: \"kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.474539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.474535    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.474731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.474591    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.475188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.475054    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.475188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.475127    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.483282 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.483246    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chjnw\" (UniqueName: \"kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.485915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.485890    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:37:27.576257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.576215    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drlr7\" (UniqueName: \"kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.576439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.576288    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.576439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.576333    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.576439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.576374    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.576439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.576404    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgqs8\" (UniqueName: \"kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.576631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.576460    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.577310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.577259    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.577428 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.577323    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.588567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.584979    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:37:27.588567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.586947    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"]
Oct 31 00:37:27.590350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.590331    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.593266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.593238    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drlr7\" (UniqueName: \"kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.595182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.595159    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 00:37:27.600435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.600074    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"]
Oct 31 00:37:27.615951 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.615862    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-drfvd\""
Oct 31 00:37:27.657754 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.657704    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"]
Oct 31 00:37:27.663383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.663354    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.679820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.679788    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-fxpxv\""
Oct 31 00:37:27.681395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681340    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681380    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681432    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.681531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681463    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc1366be-1d30-4c52-a330-4164ddae0391-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681497    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc1366be-1d30-4c52-a330-4164ddae0391-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681521    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc1366be-1d30-4c52-a330-4164ddae0391-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681550    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgqs8\" (UniqueName: \"kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.681675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681576    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681601    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681622    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsk7c\" (UniqueName: \"kubernetes.io/projected/fc1366be-1d30-4c52-a330-4164ddae0391-kube-api-access-wsk7c\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681698    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.681867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.681736    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.682837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.682208    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.682837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.682517    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.684688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.683695    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"]
Oct 31 00:37:27.686080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.685855    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:37:27.697291 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.697241    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgqs8\" (UniqueName: \"kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.779358 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.779326    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"]
Oct 31 00:37:27.782093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.782071    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:37:27.782575 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.782554    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.782648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.782590    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.782648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.782608    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.782874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.782858    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4crfb\" (UniqueName: \"kubernetes.io/projected/95d1f317-798a-49cd-8dbe-1af184a58719-kube-api-access-4crfb\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.784931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.784871    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.784931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.783514    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.784931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.783356    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.784956    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc1366be-1d30-4c52-a330-4164ddae0391-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.784986    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95d1f317-798a-49cd-8dbe-1af184a58719-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785019    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95d1f317-798a-49cd-8dbe-1af184a58719-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785047    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc1366be-1d30-4c52-a330-4164ddae0391-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785073    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc1366be-1d30-4c52-a330-4164ddae0391-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785107    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785133    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785163    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsk7c\" (UniqueName: \"kubernetes.io/projected/fc1366be-1d30-4c52-a330-4164ddae0391-kube-api-access-wsk7c\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785190    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.785478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785238    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.785478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785268    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.785478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785327    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.785478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.785367    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95d1f317-798a-49cd-8dbe-1af184a58719-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.786075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.786054    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.787017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.786994    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc1366be-1d30-4c52-a330-4164ddae0391-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.787258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.787235    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.790490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.790367    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc1366be-1d30-4c52-a330-4164ddae0391-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.792255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.792184    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w" event={"ID":"d7e2a645-1820-4ab4-8d23-05c6bba2197a","Type":"ContainerStarted","Data":"d95052cb365bc35255a7c540724e8738b6f6de0de8bebdf06a0090ecee359b53"}
Oct 31 00:37:27.792858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.792837    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc1366be-1d30-4c52-a330-4164ddae0391-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.799496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.799473    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsk7c\" (UniqueName: \"kubernetes.io/projected/fc1366be-1d30-4c52-a330-4164ddae0391-kube-api-access-wsk7c\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.804954 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.804928    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc1366be-1d30-4c52-a330-4164ddae0391-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-4f7pj\" (UID: \"fc1366be-1d30-4c52-a330-4164ddae0391\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.870164 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:37:27.870068    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:37:27.884355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.882558    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"]
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886413    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886448    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886484    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95d1f317-798a-49cd-8dbe-1af184a58719-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886512    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886538    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4crfb\" (UniqueName: \"kubernetes.io/projected/95d1f317-798a-49cd-8dbe-1af184a58719-kube-api-access-4crfb\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886563    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886592    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95d1f317-798a-49cd-8dbe-1af184a58719-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886618    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95d1f317-798a-49cd-8dbe-1af184a58719-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.886776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.886655    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.887247 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.887108    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.887759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.887527    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.887759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.887555    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.888507 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.888235    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.888736 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.888711    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95d1f317-798a-49cd-8dbe-1af184a58719-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.892122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.892091    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95d1f317-798a-49cd-8dbe-1af184a58719-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.895468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.895418    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95d1f317-798a-49cd-8dbe-1af184a58719-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.897025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.897003    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95d1f317-798a-49cd-8dbe-1af184a58719-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.898043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.898020    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4crfb\" (UniqueName: \"kubernetes.io/projected/95d1f317-798a-49cd-8dbe-1af184a58719-kube-api-access-4crfb\") pod \"openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6\" (UID: \"95d1f317-798a-49cd-8dbe-1af184a58719\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:27.906784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.906512    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:27.995315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:27.995285    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"]
Oct 31 00:37:28.003859 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:28.003822    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7714cfd1_d907_43d4_91b2_453375d32ef6.slice/crio-62b1b0896d22b958250812ba5de14fd414d4c01acf3b977f5f17efaf09d2986a WatchSource:0}: Error finding container 62b1b0896d22b958250812ba5de14fd414d4c01acf3b977f5f17efaf09d2986a: Status 404 returned error can't find the container with id 62b1b0896d22b958250812ba5de14fd414d4c01acf3b977f5f17efaf09d2986a
Oct 31 00:37:28.005555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.005535    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:28.099811 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.099718    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"]
Oct 31 00:37:28.199655 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.199565    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"]
Oct 31 00:37:28.203736 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:28.203701    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1366be_1d30_4c52_a330_4164ddae0391.slice/crio-6eb7754fae991caa58ee68be02f0968f35e5f577ae8b52eea26c045b23ba5c61 WatchSource:0}: Error finding container 6eb7754fae991caa58ee68be02f0968f35e5f577ae8b52eea26c045b23ba5c61: Status 404 returned error can't find the container with id 6eb7754fae991caa58ee68be02f0968f35e5f577ae8b52eea26c045b23ba5c61
Oct 31 00:37:28.304392 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.304353    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"]
Oct 31 00:37:28.307633 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:28.307600    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d1f317_798a_49cd_8dbe_1af184a58719.slice/crio-93a61f1a971fac058316dee76a866e587980954c7fe982f1afba3d77836a41a5 WatchSource:0}: Error finding container 93a61f1a971fac058316dee76a866e587980954c7fe982f1afba3d77836a41a5: Status 404 returned error can't find the container with id 93a61f1a971fac058316dee76a866e587980954c7fe982f1afba3d77836a41a5
Oct 31 00:37:28.801817 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.801768    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6" event={"ID":"95d1f317-798a-49cd-8dbe-1af184a58719","Type":"ContainerStarted","Data":"93a61f1a971fac058316dee76a866e587980954c7fe982f1afba3d77836a41a5"}
Oct 31 00:37:28.809191 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.809094    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" event={"ID":"fc1366be-1d30-4c52-a330-4164ddae0391","Type":"ContainerStarted","Data":"6eb7754fae991caa58ee68be02f0968f35e5f577ae8b52eea26c045b23ba5c61"}
Oct 31 00:37:28.817991 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.817894    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg" event={"ID":"7714cfd1-d907-43d4-91b2-453375d32ef6","Type":"ContainerStarted","Data":"62b1b0896d22b958250812ba5de14fd414d4c01acf3b977f5f17efaf09d2986a"}
Oct 31 00:37:28.823431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.823341    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb" event={"ID":"0b178157-7767-40f7-82ed-c0b9d8e87228","Type":"ContainerStarted","Data":"f45cacd694cfd43942cadf7c12a3a179125d74e74f66bbf59d52dd5ccce3e139"}
Oct 31 00:37:28.825713 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.825646    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v" event={"ID":"264a89e9-4c83-482d-810c-3cd43e8d8486","Type":"ContainerStarted","Data":"35aee7980c3c3b66f1e3a6e696bb6ee970d3d5d32e636b74373b2b9ddb0d2331"}
Oct 31 00:37:28.827419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:28.827394    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr" event={"ID":"e8e1e569-4787-46c9-8c7b-0732e0967b28","Type":"ContainerStarted","Data":"556ca0dd45e9656d0248f31e912c7e566949f898cf586127c948e2db2a1c739a"}
Oct 31 00:37:29.458786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.457475    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"]
Oct 31 00:37:29.462078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.461425    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.471922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.470675    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"]
Oct 31 00:37:29.605208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.605167    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.605400 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.605238    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.605400 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.605276    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh6b6\" (UniqueName: \"kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.706592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.706549    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.706822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.706626    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.706822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.706688    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh6b6\" (UniqueName: \"kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.707245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.707219    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.707888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.707866    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.730402 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.730299    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh6b6\" (UniqueName: \"kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.786410 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.786147    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:37:29.811532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.811496    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"]
Oct 31 00:37:29.816292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.816269    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:29.830595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.830557    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"]
Oct 31 00:37:29.909094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.909042    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:29.909251 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.909145    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42zgl\" (UniqueName: \"kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:29.909313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:29.909257    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.010396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.010361    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.011153 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.011000    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.011671 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.011091    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42zgl\" (UniqueName: \"kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.012339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.012291    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.012518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.012446    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.030045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.029781    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42zgl\" (UniqueName: \"kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.161543 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.161300    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:37:30.217262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.217000    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"]
Oct 31 00:37:30.234711 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:37:30.234667    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc8188f_fbc0_430b_8330_02d90c0b2a95.slice/crio-b12d3409ff68d5589c82db8b8a032ded890086dc10ade710ee4de8f833c73c19 WatchSource:0}: Error finding container b12d3409ff68d5589c82db8b8a032ded890086dc10ade710ee4de8f833c73c19: Status 404 returned error can't find the container with id b12d3409ff68d5589c82db8b8a032ded890086dc10ade710ee4de8f833c73c19
Oct 31 00:37:30.520794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.518586    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"]
Oct 31 00:37:30.862854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.861486    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv" event={"ID":"cbc8188f-fbc0-430b-8330-02d90c0b2a95","Type":"ContainerStarted","Data":"b12d3409ff68d5589c82db8b8a032ded890086dc10ade710ee4de8f833c73c19"}
Oct 31 00:37:30.868506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:30.868422    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj" event={"ID":"9a3e1098-53e3-4c59-b673-2cbafb4b0171","Type":"ContainerStarted","Data":"a6c3b4cb6c152397b7d614eecf8a00013ddd24e5b3103a68ab272a7d03a0adf3"}
Oct 31 00:37:35.955286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:35.955245    4924 generic.go:356] "Generic (PLEG): container finished" podID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerID="835c6aae7b193bc94deed0b6be1294d5dd95b04bbd1ad7b6a5a75b713408dc03" exitCode=0
Oct 31 00:37:35.955914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:35.955399    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms" event={"ID":"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd","Type":"ContainerDied","Data":"835c6aae7b193bc94deed0b6be1294d5dd95b04bbd1ad7b6a5a75b713408dc03"}
Oct 31 00:37:35.962232 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:35.960556    4924 generic.go:356] "Generic (PLEG): container finished" podID="26544eba-faec-40fb-9c51-e9017cfa111d" containerID="15e8daf59238af744c1899d0ebacd6144bacba8c8fb71bca07072473d3103e83" exitCode=0
Oct 31 00:37:35.962232 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:35.960708    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8" event={"ID":"26544eba-faec-40fb-9c51-e9017cfa111d","Type":"ContainerDied","Data":"15e8daf59238af744c1899d0ebacd6144bacba8c8fb71bca07072473d3103e83"}
Oct 31 00:37:37.883391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.883336    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:37:37.884237 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.883464    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:37:37.884613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.884579    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:37:37.985046 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.983455    4924 generic.go:356] "Generic (PLEG): container finished" podID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerID="4aa1613da66e860d5bb904313486acd17d124c9484dd0d11725700e1079be524" exitCode=0
Oct 31 00:37:37.985046 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.983800    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w" event={"ID":"d7e2a645-1820-4ab4-8d23-05c6bba2197a","Type":"ContainerDied","Data":"4aa1613da66e860d5bb904313486acd17d124c9484dd0d11725700e1079be524"}
Oct 31 00:37:38.000189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.999080    4924 generic.go:356] "Generic (PLEG): container finished" podID="26544eba-faec-40fb-9c51-e9017cfa111d" containerID="ad14a4a4a2aa8bd7336e6231778ae5a694836b52f97dbc6c373c62eedfaaaade" exitCode=0
Oct 31 00:37:38.000189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:37.999288    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8" event={"ID":"26544eba-faec-40fb-9c51-e9017cfa111d","Type":"ContainerDied","Data":"ad14a4a4a2aa8bd7336e6231778ae5a694836b52f97dbc6c373c62eedfaaaade"}
Oct 31 00:37:38.422552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:38.422477    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:37:38.422684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:38.422665    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:37:38.422762 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:38.422718    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:37:39.022870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:39.022829    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" event={"ID":"fc1366be-1d30-4c52-a330-4164ddae0391","Type":"ContainerStarted","Data":"35428c9a70e8aba513c035e4236e332f199b7fc157001cf6807b6e4fa2e826e9"}
Oct 31 00:37:39.036902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:39.036802    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6" event={"ID":"95d1f317-798a-49cd-8dbe-1af184a58719","Type":"ContainerStarted","Data":"3fecc8914021c076897bf6d613cd029c01953885dd43586101ec3b43a838a473"}
Oct 31 00:37:39.045354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:39.045301    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podStartSLOduration=1.830370925 podStartE2EDuration="12.045286031s" totalImagesPullingTime="10.214915106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:39.043864459 +0000 UTC m=+259.414648261" watchObservedRunningTime="2026-10-31 00:37:39.045286031 +0000 UTC m=+259.416069820"
Oct 31 00:37:39.065541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:39.065482    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6" podStartSLOduration=2.5111274139999997 podStartE2EDuration="12.065467023s" totalImagesPullingTime="9.554339609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:39.065152472 +0000 UTC m=+259.435936262" watchObservedRunningTime="2026-10-31 00:37:39.065467023 +0000 UTC m=+259.436250793"
Oct 31 00:37:39.908724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:39.908700    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:39.913057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:39.913032    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:40.006817 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.006783    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:40.012727 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.012699    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:40.047144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.047107    4924 generic.go:356] "Generic (PLEG): container finished" podID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerID="1d71315f0407f1a2a2002a58a16ca893f100f4f5572101fef8fa0b53f542433c" exitCode=0
Oct 31 00:37:40.047640 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.047236    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg" event={"ID":"7714cfd1-d907-43d4-91b2-453375d32ef6","Type":"ContainerDied","Data":"1d71315f0407f1a2a2002a58a16ca893f100f4f5572101fef8fa0b53f542433c"}
Oct 31 00:37:40.050651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.050627    4924 generic.go:356] "Generic (PLEG): container finished" podID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerID="1f03b4f2963df4e881ec7232db9ec167ccd2a8a3188a2242d895010b75f38246" exitCode=0
Oct 31 00:37:40.051041 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.050934    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb" event={"ID":"0b178157-7767-40f7-82ed-c0b9d8e87228","Type":"ContainerDied","Data":"1f03b4f2963df4e881ec7232db9ec167ccd2a8a3188a2242d895010b75f38246"}
Oct 31 00:37:40.056160 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.056131    4924 generic.go:356] "Generic (PLEG): container finished" podID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerID="f3bfe392fec7b7040e06876df45b7902baabe982bcd7f451ca99244c647d6b86" exitCode=0
Oct 31 00:37:40.056266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.056207    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v" event={"ID":"264a89e9-4c83-482d-810c-3cd43e8d8486","Type":"ContainerDied","Data":"f3bfe392fec7b7040e06876df45b7902baabe982bcd7f451ca99244c647d6b86"}
Oct 31 00:37:40.059997 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.059895    4924 generic.go:356] "Generic (PLEG): container finished" podID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerID="14b49cfc19bc458e752ae77d426479a88dffd75884dec0e6c2d8339a0cd70df8" exitCode=0
Oct 31 00:37:40.061330 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.061297    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr" event={"ID":"e8e1e569-4787-46c9-8c7b-0732e0967b28","Type":"ContainerDied","Data":"14b49cfc19bc458e752ae77d426479a88dffd75884dec0e6c2d8339a0cd70df8"}
Oct 31 00:37:40.061432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.061335    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:40.061824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.061804    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:40.063779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.063585    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6"
Oct 31 00:37:40.063779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:40.063734    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:37:42.074549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.074515    4924 generic.go:356] "Generic (PLEG): container finished" podID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerID="d6b5da6f0ac4bcd30cc102278262810df6debc2600bc8c35bf842a0318572f25" exitCode=0
Oct 31 00:37:42.075012 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.074573    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms" event={"ID":"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd","Type":"ContainerDied","Data":"d6b5da6f0ac4bcd30cc102278262810df6debc2600bc8c35bf842a0318572f25"}
Oct 31 00:37:42.076403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.076372    4924 generic.go:356] "Generic (PLEG): container finished" podID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerID="bcd232950e74478618f3fb916c1548563afbeb2f830d24816636f2dc4bb55b9e" exitCode=0
Oct 31 00:37:42.076644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.076511    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv" event={"ID":"cbc8188f-fbc0-430b-8330-02d90c0b2a95","Type":"ContainerDied","Data":"bcd232950e74478618f3fb916c1548563afbeb2f830d24816636f2dc4bb55b9e"}
Oct 31 00:37:42.078239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.078217    4924 generic.go:356] "Generic (PLEG): container finished" podID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerID="3db2aca0b52f80964690849b8789346b0bcce65ea7639202f8475678a74ad627" exitCode=0
Oct 31 00:37:42.078352 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.078329    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj" event={"ID":"9a3e1098-53e3-4c59-b673-2cbafb4b0171","Type":"ContainerDied","Data":"3db2aca0b52f80964690849b8789346b0bcce65ea7639202f8475678a74ad627"}
Oct 31 00:37:42.081001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.080971    4924 generic.go:356] "Generic (PLEG): container finished" podID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerID="73cb39275abf801eb87e9d24c0ca3d775107f79caa566bd384c0ae4f1f5b12a9" exitCode=0
Oct 31 00:37:42.081181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.081160    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg" event={"ID":"7714cfd1-d907-43d4-91b2-453375d32ef6","Type":"ContainerDied","Data":"73cb39275abf801eb87e9d24c0ca3d775107f79caa566bd384c0ae4f1f5b12a9"}
Oct 31 00:37:42.083297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.083272    4924 generic.go:356] "Generic (PLEG): container finished" podID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerID="780b3e9bc8a65341e8f5a48910d3fbae9bcf9beb16074a0b050ae66cce6b4e40" exitCode=0
Oct 31 00:37:42.083539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.083517    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb" event={"ID":"0b178157-7767-40f7-82ed-c0b9d8e87228","Type":"ContainerDied","Data":"780b3e9bc8a65341e8f5a48910d3fbae9bcf9beb16074a0b050ae66cce6b4e40"}
Oct 31 00:37:42.086358 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.086334    4924 generic.go:356] "Generic (PLEG): container finished" podID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerID="e743ce4117773fbc233bb15ffce10bae7611f015c7911e5e9f084c7a55c16962" exitCode=0
Oct 31 00:37:42.086597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.086579    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v" event={"ID":"264a89e9-4c83-482d-810c-3cd43e8d8486","Type":"ContainerDied","Data":"e743ce4117773fbc233bb15ffce10bae7611f015c7911e5e9f084c7a55c16962"}
Oct 31 00:37:42.089373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.089353    4924 generic.go:356] "Generic (PLEG): container finished" podID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerID="74add50088701d1a6eb44680c23655167c57880415c7b76132cbd81f9f442a3e" exitCode=0
Oct 31 00:37:42.089457 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:42.089437    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr" event={"ID":"e8e1e569-4787-46c9-8c7b-0732e0967b28","Type":"ContainerDied","Data":"74add50088701d1a6eb44680c23655167c57880415c7b76132cbd81f9f442a3e"}
Oct 31 00:37:43.104684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:43.104645    4924 generic.go:356] "Generic (PLEG): container finished" podID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerID="46bba5572887a47b4f12802168c8e1264486d969fb8ea779a453508a5792dfc7" exitCode=0
Oct 31 00:37:43.105266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:43.104779    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv" event={"ID":"cbc8188f-fbc0-430b-8330-02d90c0b2a95","Type":"ContainerDied","Data":"46bba5572887a47b4f12802168c8e1264486d969fb8ea779a453508a5792dfc7"}
Oct 31 00:37:43.112331 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:43.111569    4924 generic.go:356] "Generic (PLEG): container finished" podID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerID="523f28446950f44775580407cd9329bf3327cc4068bf3363b79d659a14a6d62c" exitCode=0
Oct 31 00:37:43.112331 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:43.111652    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w" event={"ID":"d7e2a645-1820-4ab4-8d23-05c6bba2197a","Type":"ContainerDied","Data":"523f28446950f44775580407cd9329bf3327cc4068bf3363b79d659a14a6d62c"}
Oct 31 00:37:43.119489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:43.118719    4924 generic.go:356] "Generic (PLEG): container finished" podID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerID="90458f6aee878b27409a305831281ec778756881c2565ce4f18002ab295a5561" exitCode=0
Oct 31 00:37:43.119489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:43.118840    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj" event={"ID":"9a3e1098-53e3-4c59-b673-2cbafb4b0171","Type":"ContainerDied","Data":"90458f6aee878b27409a305831281ec778756881c2565ce4f18002ab295a5561"}
Oct 31 00:37:56.283642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:56.283598    4924 generic.go:356] "Generic (PLEG): container finished" podID="26544eba-faec-40fb-9c51-e9017cfa111d" containerID="e406506bf8c7aaef16126fc7100acea6bdd625054d6bd7d8bfd9a082314a75f5" exitCode=0
Oct 31 00:37:56.284263 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:56.283672    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8" event={"ID":"26544eba-faec-40fb-9c51-e9017cfa111d","Type":"ContainerDied","Data":"e406506bf8c7aaef16126fc7100acea6bdd625054d6bd7d8bfd9a082314a75f5"}
Oct 31 00:37:57.550824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.550696    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:37:57.617642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.616599    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle\") pod \"26544eba-faec-40fb-9c51-e9017cfa111d\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") "
Oct 31 00:37:57.617642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.616711    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util\") pod \"26544eba-faec-40fb-9c51-e9017cfa111d\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") "
Oct 31 00:37:57.617642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.616782    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m\" (UniqueName: \"kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m\") pod \"26544eba-faec-40fb-9c51-e9017cfa111d\" (UID: \"26544eba-faec-40fb-9c51-e9017cfa111d\") "
Oct 31 00:37:57.623216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.622825    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle" pod "26544eba-faec-40fb-9c51-e9017cfa111d" (UID: "26544eba-faec-40fb-9c51-e9017cfa111d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:37:57.623216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.623135    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m" pod "26544eba-faec-40fb-9c51-e9017cfa111d" (UID: "26544eba-faec-40fb-9c51-e9017cfa111d"). InnerVolumeSpecName "kube-api-access-mnv4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:37:57.627182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.627130    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util" pod "26544eba-faec-40fb-9c51-e9017cfa111d" (UID: "26544eba-faec-40fb-9c51-e9017cfa111d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:37:57.718396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.718310    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:37:57.718396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.718353    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26544eba-faec-40fb-9c51-e9017cfa111d-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:37:57.718396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:57.718369    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnv4m\" (UniqueName: \"kubernetes.io/projected/26544eba-faec-40fb-9c51-e9017cfa111d-kube-api-access-mnv4m\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:37:57.880585 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:37:57.880467    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:37:58.308794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:58.308737    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8" event={"ID":"26544eba-faec-40fb-9c51-e9017cfa111d","Type":"ContainerDied","Data":"ca631b63d96adf4ef81ab2faff12c59bfbdd6cc37ff0d1f3f6d20303e42ad93a"}
Oct 31 00:37:58.308984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:58.308819    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca631b63d96adf4ef81ab2faff12c59bfbdd6cc37ff0d1f3f6d20303e42ad93a"
Oct 31 00:37:58.308984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:37:58.308897    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71542w8"
Oct 31 00:38:01.333264 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.333222    4924 generic.go:356] "Generic (PLEG): container finished" podID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerID="3c229943bd3913f48a33f91ae0c9b872f8f8ef5006355ab084330d01b309d1c6" exitCode=0
Oct 31 00:38:01.333715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.333310    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms" event={"ID":"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd","Type":"ContainerDied","Data":"3c229943bd3913f48a33f91ae0c9b872f8f8ef5006355ab084330d01b309d1c6"}
Oct 31 00:38:01.335975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.335933    4924 generic.go:356] "Generic (PLEG): container finished" podID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerID="a792c7db44d22de6e3d9e7ab1ff6463d99b929a738b801af73fee626142c9f03" exitCode=0
Oct 31 00:38:01.336117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.336018    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg" event={"ID":"7714cfd1-d907-43d4-91b2-453375d32ef6","Type":"ContainerDied","Data":"a792c7db44d22de6e3d9e7ab1ff6463d99b929a738b801af73fee626142c9f03"}
Oct 31 00:38:01.338737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.338618    4924 generic.go:356] "Generic (PLEG): container finished" podID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerID="6b29afb02da1072d164eae123995185dc10a092639d2b724dd488d4b2b33c3b4" exitCode=0
Oct 31 00:38:01.338737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.338651    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb" event={"ID":"0b178157-7767-40f7-82ed-c0b9d8e87228","Type":"ContainerDied","Data":"6b29afb02da1072d164eae123995185dc10a092639d2b724dd488d4b2b33c3b4"}
Oct 31 00:38:01.341159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.341133    4924 generic.go:356] "Generic (PLEG): container finished" podID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerID="510d92fe124d2152b351ec40f22f47df6632c076b2df75f475d72fafed727499" exitCode=0
Oct 31 00:38:01.341292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.341207    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v" event={"ID":"264a89e9-4c83-482d-810c-3cd43e8d8486","Type":"ContainerDied","Data":"510d92fe124d2152b351ec40f22f47df6632c076b2df75f475d72fafed727499"}
Oct 31 00:38:01.343656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.343634    4924 generic.go:356] "Generic (PLEG): container finished" podID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerID="e5a576c8b030521f9ced875ae81ecca01fe29528d0bf9b142bf16eb37398db53" exitCode=0
Oct 31 00:38:01.343774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:01.343684    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr" event={"ID":"e8e1e569-4787-46c9-8c7b-0732e0967b28","Type":"ContainerDied","Data":"e5a576c8b030521f9ced875ae81ecca01fe29528d0bf9b142bf16eb37398db53"}
Oct 31 00:38:02.350506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.350468    4924 generic.go:356] "Generic (PLEG): container finished" podID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerID="a4f91f2b94bfadd07681be9dd080447795d98728ee500e44bf27a1c523693f4f" exitCode=0
Oct 31 00:38:02.350954 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.350558    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv" event={"ID":"cbc8188f-fbc0-430b-8330-02d90c0b2a95","Type":"ContainerDied","Data":"a4f91f2b94bfadd07681be9dd080447795d98728ee500e44bf27a1c523693f4f"}
Oct 31 00:38:02.352580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.352556    4924 generic.go:356] "Generic (PLEG): container finished" podID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerID="701d9a0559034e333fd3554c7bc9ca09ffe1e2e8180d5d58bbb2eb95d388ef0d" exitCode=0
Oct 31 00:38:02.352656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.352634    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w" event={"ID":"d7e2a645-1820-4ab4-8d23-05c6bba2197a","Type":"ContainerDied","Data":"701d9a0559034e333fd3554c7bc9ca09ffe1e2e8180d5d58bbb2eb95d388ef0d"}
Oct 31 00:38:02.354778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.354732    4924 generic.go:356] "Generic (PLEG): container finished" podID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerID="b4fc10d5a5736a154e159570a06a443732a00163ad383bea0468ca5be3d18898" exitCode=0
Oct 31 00:38:02.354890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.354778    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj" event={"ID":"9a3e1098-53e3-4c59-b673-2cbafb4b0171","Type":"ContainerDied","Data":"b4fc10d5a5736a154e159570a06a443732a00163ad383bea0468ca5be3d18898"}
Oct 31 00:38:02.529215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.529177    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:38:02.555669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.555633    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:38:02.602781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.602733    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:38:02.609224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.609198    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:38:02.612893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.612872    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:38:02.664544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664513    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts\" (UniqueName: \"kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts\") pod \"e8e1e569-4787-46c9-8c7b-0732e0967b28\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") "
Oct 31 00:38:02.664544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664550    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util\") pod \"7714cfd1-d907-43d4-91b2-453375d32ef6\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") "
Oct 31 00:38:02.664816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664578    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util\") pod \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") "
Oct 31 00:38:02.664816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664625    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7\" (UniqueName: \"kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7\") pod \"7714cfd1-d907-43d4-91b2-453375d32ef6\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") "
Oct 31 00:38:02.664816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664667    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util\") pod \"e8e1e569-4787-46c9-8c7b-0732e0967b28\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") "
Oct 31 00:38:02.664816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664716    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw\" (UniqueName: \"kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw\") pod \"0b178157-7767-40f7-82ed-c0b9d8e87228\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") "
Oct 31 00:38:02.664816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664762    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8\" (UniqueName: \"kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8\") pod \"264a89e9-4c83-482d-810c-3cd43e8d8486\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664844    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv\" (UniqueName: \"kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv\") pod \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664871    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle\") pod \"0b178157-7767-40f7-82ed-c0b9d8e87228\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664899    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util\") pod \"264a89e9-4c83-482d-810c-3cd43e8d8486\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664928    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util\") pod \"0b178157-7767-40f7-82ed-c0b9d8e87228\" (UID: \"0b178157-7767-40f7-82ed-c0b9d8e87228\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.664957    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle\") pod \"7714cfd1-d907-43d4-91b2-453375d32ef6\" (UID: \"7714cfd1-d907-43d4-91b2-453375d32ef6\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.665008    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle\") pod \"e8e1e569-4787-46c9-8c7b-0732e0967b28\" (UID: \"e8e1e569-4787-46c9-8c7b-0732e0967b28\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.665035    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle\") pod \"264a89e9-4c83-482d-810c-3cd43e8d8486\" (UID: \"264a89e9-4c83-482d-810c-3cd43e8d8486\") "
Oct 31 00:38:02.665095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.665078    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle\") pod \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\" (UID: \"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd\") "
Oct 31 00:38:02.669415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.665822    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle" pod "c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" (UID: "c0d8cf0e-9834-41ea-b84f-fff476a6e5dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.669415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.666314    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle" pod "7714cfd1-d907-43d4-91b2-453375d32ef6" (UID: "7714cfd1-d907-43d4-91b2-453375d32ef6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.669415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.666932    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle" pod "e8e1e569-4787-46c9-8c7b-0732e0967b28" (UID: "e8e1e569-4787-46c9-8c7b-0732e0967b28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.669415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.667519    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle" pod "264a89e9-4c83-482d-810c-3cd43e8d8486" (UID: "264a89e9-4c83-482d-810c-3cd43e8d8486"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.669759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.669504    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv" pod "c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" (UID: "c0d8cf0e-9834-41ea-b84f-fff476a6e5dd"). InnerVolumeSpecName "kube-api-access-rnzdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:02.669829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.669793    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8" pod "264a89e9-4c83-482d-810c-3cd43e8d8486" (UID: "264a89e9-4c83-482d-810c-3cd43e8d8486"). InnerVolumeSpecName "kube-api-access-jgqs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:02.671799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.669981    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle" pod "0b178157-7767-40f7-82ed-c0b9d8e87228" (UID: "0b178157-7767-40f7-82ed-c0b9d8e87228"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.671799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.670218    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util" pod "c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" (UID: "c0d8cf0e-9834-41ea-b84f-fff476a6e5dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.672682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.672660    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util" pod "7714cfd1-d907-43d4-91b2-453375d32ef6" (UID: "7714cfd1-d907-43d4-91b2-453375d32ef6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.674477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.674457    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7" pod "7714cfd1-d907-43d4-91b2-453375d32ef6" (UID: "7714cfd1-d907-43d4-91b2-453375d32ef6"). InnerVolumeSpecName "kube-api-access-drlr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:02.675976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.675951    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util" pod "0b178157-7767-40f7-82ed-c0b9d8e87228" (UID: "0b178157-7767-40f7-82ed-c0b9d8e87228"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.677870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.677834    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util" pod "264a89e9-4c83-482d-810c-3cd43e8d8486" (UID: "264a89e9-4c83-482d-810c-3cd43e8d8486"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.677999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.677921    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw" pod "0b178157-7767-40f7-82ed-c0b9d8e87228" (UID: "0b178157-7767-40f7-82ed-c0b9d8e87228"). InnerVolumeSpecName "kube-api-access-chjnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:02.678624 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.678597    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts" pod "e8e1e569-4787-46c9-8c7b-0732e0967b28" (UID: "e8e1e569-4787-46c9-8c7b-0732e0967b28"). InnerVolumeSpecName "kube-api-access-b5zts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:02.681048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.681028    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util" pod "e8e1e569-4787-46c9-8c7b-0732e0967b28" (UID: "e8e1e569-4787-46c9-8c7b-0732e0967b28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:02.766095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766060    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766090    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766099    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766107    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766115    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/264a89e9-4c83-482d-810c-3cd43e8d8486-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766123    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766132    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5zts\" (UniqueName: \"kubernetes.io/projected/e8e1e569-4787-46c9-8c7b-0732e0967b28-kube-api-access-b5zts\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766140    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7714cfd1-d907-43d4-91b2-453375d32ef6-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766148    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766159    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drlr7\" (UniqueName: \"kubernetes.io/projected/7714cfd1-d907-43d4-91b2-453375d32ef6-kube-api-access-drlr7\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766170    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e1e569-4787-46c9-8c7b-0732e0967b28-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766179    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chjnw\" (UniqueName: \"kubernetes.io/projected/0b178157-7767-40f7-82ed-c0b9d8e87228-kube-api-access-chjnw\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766187    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgqs8\" (UniqueName: \"kubernetes.io/projected/264a89e9-4c83-482d-810c-3cd43e8d8486-kube-api-access-jgqs8\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766197    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnzdv\" (UniqueName: \"kubernetes.io/projected/c0d8cf0e-9834-41ea-b84f-fff476a6e5dd-kube-api-access-rnzdv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:02.766326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:02.766206    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b178157-7767-40f7-82ed-c0b9d8e87228-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.361135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.361103    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms"
Oct 31 00:38:03.361558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.361133    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cx8ms" event={"ID":"c0d8cf0e-9834-41ea-b84f-fff476a6e5dd","Type":"ContainerDied","Data":"6ec46af0e3a979e81ee81605c7fd69bc0f59bcc233f0dc87eb6d1fccc7056061"}
Oct 31 00:38:03.361558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.361179    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ec46af0e3a979e81ee81605c7fd69bc0f59bcc233f0dc87eb6d1fccc7056061"
Oct 31 00:38:03.362975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.362946    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg" event={"ID":"7714cfd1-d907-43d4-91b2-453375d32ef6","Type":"ContainerDied","Data":"62b1b0896d22b958250812ba5de14fd414d4c01acf3b977f5f17efaf09d2986a"}
Oct 31 00:38:03.362975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.362958    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct5vcg"
Oct 31 00:38:03.363165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.362979    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62b1b0896d22b958250812ba5de14fd414d4c01acf3b977f5f17efaf09d2986a"
Oct 31 00:38:03.365216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.365196    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb" event={"ID":"0b178157-7767-40f7-82ed-c0b9d8e87228","Type":"ContainerDied","Data":"f45cacd694cfd43942cadf7c12a3a179125d74e74f66bbf59d52dd5ccce3e139"}
Oct 31 00:38:03.365216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.365208    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847arhsbb"
Oct 31 00:38:03.365383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.365229    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f45cacd694cfd43942cadf7c12a3a179125d74e74f66bbf59d52dd5ccce3e139"
Oct 31 00:38:03.367317 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.367296    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v" event={"ID":"264a89e9-4c83-482d-810c-3cd43e8d8486","Type":"ContainerDied","Data":"35aee7980c3c3b66f1e3a6e696bb6ee970d3d5d32e636b74373b2b9ddb0d2331"}
Oct 31 00:38:03.367445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.367333    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35aee7980c3c3b66f1e3a6e696bb6ee970d3d5d32e636b74373b2b9ddb0d2331"
Oct 31 00:38:03.367445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.367302    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68lg8v"
Oct 31 00:38:03.369511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.369481    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr"
Oct 31 00:38:03.369511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.369493    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b9qkr" event={"ID":"e8e1e569-4787-46c9-8c7b-0732e0967b28","Type":"ContainerDied","Data":"556ca0dd45e9656d0248f31e912c7e566949f898cf586127c948e2db2a1c739a"}
Oct 31 00:38:03.369631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.369528    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="556ca0dd45e9656d0248f31e912c7e566949f898cf586127c948e2db2a1c739a"
Oct 31 00:38:03.517983 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.517954    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:38:03.532103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.532075    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:38:03.565730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.565698    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:38:03.573660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.573630    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util\") pod \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") "
Oct 31 00:38:03.573857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.573717    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr\" (UniqueName: \"kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr\") pod \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") "
Oct 31 00:38:03.573857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.573738    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle\") pod \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") "
Oct 31 00:38:03.573857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.573780    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle\") pod \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") "
Oct 31 00:38:03.573857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.573832    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util\") pod \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\" (UID: \"d7e2a645-1820-4ab4-8d23-05c6bba2197a\") "
Oct 31 00:38:03.574063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.573871    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl\" (UniqueName: \"kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl\") pod \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\" (UID: \"9a3e1098-53e3-4c59-b673-2cbafb4b0171\") "
Oct 31 00:38:03.574482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.574452    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle" pod "9a3e1098-53e3-4c59-b673-2cbafb4b0171" (UID: "9a3e1098-53e3-4c59-b673-2cbafb4b0171"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:03.574651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.574608    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle" pod "d7e2a645-1820-4ab4-8d23-05c6bba2197a" (UID: "d7e2a645-1820-4ab4-8d23-05c6bba2197a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:03.576578 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.576553    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl" pod "9a3e1098-53e3-4c59-b673-2cbafb4b0171" (UID: "9a3e1098-53e3-4c59-b673-2cbafb4b0171"). InnerVolumeSpecName "kube-api-access-42zgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:03.576685 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.576589    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr" pod "d7e2a645-1820-4ab4-8d23-05c6bba2197a" (UID: "d7e2a645-1820-4ab4-8d23-05c6bba2197a"). InnerVolumeSpecName "kube-api-access-wjvqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:03.579527 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.579501    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util" pod "9a3e1098-53e3-4c59-b673-2cbafb4b0171" (UID: "9a3e1098-53e3-4c59-b673-2cbafb4b0171"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:03.579621 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.579600    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util" pod "d7e2a645-1820-4ab4-8d23-05c6bba2197a" (UID: "d7e2a645-1820-4ab4-8d23-05c6bba2197a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:03.674679 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.674649    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util\") pod \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") "
Oct 31 00:38:03.674869 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.674701    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6\" (UniqueName: \"kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6\") pod \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") "
Oct 31 00:38:03.674869 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.674796    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle\") pod \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\" (UID: \"cbc8188f-fbc0-430b-8330-02d90c0b2a95\") "
Oct 31 00:38:03.674995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.674977    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.675040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.675003    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjvqr\" (UniqueName: \"kubernetes.io/projected/d7e2a645-1820-4ab4-8d23-05c6bba2197a-kube-api-access-wjvqr\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.675040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.675019    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a3e1098-53e3-4c59-b673-2cbafb4b0171-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.675040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.675032    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.675145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.675044    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7e2a645-1820-4ab4-8d23-05c6bba2197a-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.675145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.675057    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42zgl\" (UniqueName: \"kubernetes.io/projected/9a3e1098-53e3-4c59-b673-2cbafb4b0171-kube-api-access-42zgl\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.675212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.675194    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle" pod "cbc8188f-fbc0-430b-8330-02d90c0b2a95" (UID: "cbc8188f-fbc0-430b-8330-02d90c0b2a95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:03.677098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.677069    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6" pod "cbc8188f-fbc0-430b-8330-02d90c0b2a95" (UID: "cbc8188f-fbc0-430b-8330-02d90c0b2a95"). InnerVolumeSpecName "kube-api-access-rh6b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:03.680356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.680332    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util" pod "cbc8188f-fbc0-430b-8330-02d90c0b2a95" (UID: "cbc8188f-fbc0-430b-8330-02d90c0b2a95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:03.776224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.776205    4924 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-bundle\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.776224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.776225    4924 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbc8188f-fbc0-430b-8330-02d90c0b2a95-util\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:03.776345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:03.776234    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh6b6\" (UniqueName: \"kubernetes.io/projected/cbc8188f-fbc0-430b-8330-02d90c0b2a95-kube-api-access-rh6b6\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:04.376284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.376255    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv"
Oct 31 00:38:04.376767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.376253    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644764rv" event={"ID":"cbc8188f-fbc0-430b-8330-02d90c0b2a95","Type":"ContainerDied","Data":"b12d3409ff68d5589c82db8b8a032ded890086dc10ade710ee4de8f833c73c19"}
Oct 31 00:38:04.376767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.376367    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b12d3409ff68d5589c82db8b8a032ded890086dc10ade710ee4de8f833c73c19"
Oct 31 00:38:04.378361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.378325    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w"
Oct 31 00:38:04.378361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.378334    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czd82w" event={"ID":"d7e2a645-1820-4ab4-8d23-05c6bba2197a","Type":"ContainerDied","Data":"d95052cb365bc35255a7c540724e8738b6f6de0de8bebdf06a0090ecee359b53"}
Oct 31 00:38:04.378712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.378374    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d95052cb365bc35255a7c540724e8738b6f6de0de8bebdf06a0090ecee359b53"
Oct 31 00:38:04.380523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.380506    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj"
Oct 31 00:38:04.380614 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.380541    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7146mdj" event={"ID":"9a3e1098-53e3-4c59-b673-2cbafb4b0171","Type":"ContainerDied","Data":"a6c3b4cb6c152397b7d614eecf8a00013ddd24e5b3103a68ab272a7d03a0adf3"}
Oct 31 00:38:04.380614 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:04.380576    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6c3b4cb6c152397b7d614eecf8a00013ddd24e5b3103a68ab272a7d03a0adf3"
Oct 31 00:38:12.039058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039020    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"]
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039316    4924 cpu_manager.go:414] "containerMap: removing container" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039325    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039335    4924 cpu_manager.go:414] "containerMap: removing container" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039340    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039348    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039352    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039359    4924 cpu_manager.go:414] "containerMap: removing container" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039363    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039372    4924 cpu_manager.go:414] "containerMap: removing container" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039376    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039385    4924 cpu_manager.go:414] "containerMap: removing container" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039390    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039395    4924 cpu_manager.go:414] "containerMap: removing container" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039399    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039406    4924 cpu_manager.go:414] "containerMap: removing container" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039410    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039416    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039420    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039425    4924 cpu_manager.go:414] "containerMap: removing container" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039429    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039437    4924 cpu_manager.go:414] "containerMap: removing container" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039442    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039447    4924 cpu_manager.go:414] "containerMap: removing container" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039451    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039458    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039462    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039470    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039475    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039480    4924 cpu_manager.go:414] "containerMap: removing container" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039485    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039497    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039505    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="pull"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039516    4924 cpu_manager.go:414] "containerMap: removing container" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039523    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="util"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039529    4924 cpu_manager.go:414] "containerMap: removing container" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039534    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039541    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="extract"
Oct 31 00:38:12.039529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039545    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039554    4924 cpu_manager.go:414] "containerMap: removing container" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="util"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039559    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="util"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039566    4924 cpu_manager.go:414] "containerMap: removing container" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="util"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039570    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="util"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039575    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="util"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039579    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="util"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039584    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039588    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039594    4924 cpu_manager.go:414] "containerMap: removing container" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="pull"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039598    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="pull"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039604    4924 cpu_manager.go:414] "containerMap: removing container" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039608    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039613    4924 cpu_manager.go:414] "containerMap: removing container" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039617    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039622    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039625    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039678    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="26544eba-faec-40fb-9c51-e9017cfa111d" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039686    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="264a89e9-4c83-482d-810c-3cd43e8d8486" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039693    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbc8188f-fbc0-430b-8330-02d90c0b2a95" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039698    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0d8cf0e-9834-41ea-b84f-fff476a6e5dd" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039703    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a3e1098-53e3-4c59-b673-2cbafb4b0171" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039708    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="7714cfd1-d907-43d4-91b2-453375d32ef6" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039730    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b178157-7767-40f7-82ed-c0b9d8e87228" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039736    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7e2a645-1820-4ab4-8d23-05c6bba2197a" containerName="extract"
Oct 31 00:38:12.040589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.039756    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8e1e569-4787-46c9-8c7b-0732e0967b28" containerName="extract"
Oct 31 00:38:12.043782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.043764    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.056643 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.056615    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 00:38:12.056968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.056951    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cbhxv\""
Oct 31 00:38:12.056968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.056962    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 00:38:12.066305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.066273    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"]
Oct 31 00:38:12.148841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.148797    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msxsp\" (UniqueName: \"kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp\") pod \"kuadrant-operator-controller-manager-cf8469d7b-2qnmb\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.149037 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.148872    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-2qnmb\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.249559 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.249515    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msxsp\" (UniqueName: \"kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp\") pod \"kuadrant-operator-controller-manager-cf8469d7b-2qnmb\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.249771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.249580    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-2qnmb\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.249956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.249940    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-2qnmb\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.260297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.260271    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msxsp\" (UniqueName: \"kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp\") pod \"kuadrant-operator-controller-manager-cf8469d7b-2qnmb\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.356518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.356430    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:12.651199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:12.651170    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"]
Oct 31 00:38:12.653701 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:12.653662    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6971fbe7_353d_47dd_8dec_cefb775182d6.slice/crio-b8bb4a3890292f8e7d68b0f0e61b465d5dc0e02d74955a50ea058f0c6dc9258c WatchSource:0}: Error finding container b8bb4a3890292f8e7d68b0f0e61b465d5dc0e02d74955a50ea058f0c6dc9258c: Status 404 returned error can't find the container with id b8bb4a3890292f8e7d68b0f0e61b465d5dc0e02d74955a50ea058f0c6dc9258c
Oct 31 00:38:13.422580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.422547    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb" event={"ID":"6971fbe7-353d-47dd-8dec-cefb775182d6","Type":"ContainerStarted","Data":"b8bb4a3890292f8e7d68b0f0e61b465d5dc0e02d74955a50ea058f0c6dc9258c"}
Oct 31 00:38:13.548081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.548042    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"]
Oct 31 00:38:13.553039 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.553008    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.565548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.565523    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 00:38:13.565668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.565598    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 00:38:13.565934 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.565914    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-j6mcx\""
Oct 31 00:38:13.565985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.565937    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 00:38:13.568335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.568315    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 00:38:13.568426 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.568377    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 00:38:13.582850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.582825    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"]
Oct 31 00:38:13.662633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.662600    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.662836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.662650    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c25d6a67-6e6f-4703-8bd4-4374035bffbf-manifests\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.662836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.662765    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.662836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.662821    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-metrics-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.663023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.662940    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c25d6a67-6e6f-4703-8bd4-4374035bffbf-tmp\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.663023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.662977    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnlrg\" (UniqueName: \"kubernetes.io/projected/c25d6a67-6e6f-4703-8bd4-4374035bffbf-kube-api-access-jnlrg\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.763942    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c25d6a67-6e6f-4703-8bd4-4374035bffbf-tmp\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.763978    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnlrg\" (UniqueName: \"kubernetes.io/projected/c25d6a67-6e6f-4703-8bd4-4374035bffbf-kube-api-access-jnlrg\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.764158    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.764202    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c25d6a67-6e6f-4703-8bd4-4374035bffbf-manifests\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.764225    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.764265    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-metrics-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.764616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.764595    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c25d6a67-6e6f-4703-8bd4-4374035bffbf-manifests\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.766282 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.766259    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c25d6a67-6e6f-4703-8bd4-4374035bffbf-tmp\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.766663 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.766646    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.766713 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.766691    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.766767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.766691    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c25d6a67-6e6f-4703-8bd4-4374035bffbf-metrics-cert\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.774112 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.774087    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnlrg\" (UniqueName: \"kubernetes.io/projected/c25d6a67-6e6f-4703-8bd4-4374035bffbf-kube-api-access-jnlrg\") pod \"opendatahub-operator-controller-manager-54459f848c-4w4wn\" (UID: \"c25d6a67-6e6f-4703-8bd4-4374035bffbf\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:13.869059 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:13.869002    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:14.177713 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:14.177666    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"]
Oct 31 00:38:14.181337 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:14.181298    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25d6a67_6e6f_4703_8bd4_4374035bffbf.slice/crio-a195c7df6892881bc6bf994c65342f2d01210317297246afc7c5f7ce7c8d89d2 WatchSource:0}: Error finding container a195c7df6892881bc6bf994c65342f2d01210317297246afc7c5f7ce7c8d89d2: Status 404 returned error can't find the container with id a195c7df6892881bc6bf994c65342f2d01210317297246afc7c5f7ce7c8d89d2
Oct 31 00:38:14.428200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:14.428110    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn" event={"ID":"c25d6a67-6e6f-4703-8bd4-4374035bffbf","Type":"ContainerStarted","Data":"a195c7df6892881bc6bf994c65342f2d01210317297246afc7c5f7ce7c8d89d2"}
Oct 31 00:38:15.741948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.741909    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"]
Oct 31 00:38:15.745596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.745572    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:15.750783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.750759    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-98hdv\""
Oct 31 00:38:15.755860 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.755834    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 31 00:38:15.756065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.756042    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"]
Oct 31 00:38:15.883391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.883346    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnm2v\" (UniqueName: \"kubernetes.io/projected/3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08-kube-api-access-tnm2v\") pod \"dns-operator-controller-manager-8557cbdb84-ppnqs\" (UID: \"3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:15.984488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.984446    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnm2v\" (UniqueName: \"kubernetes.io/projected/3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08-kube-api-access-tnm2v\") pod \"dns-operator-controller-manager-8557cbdb84-ppnqs\" (UID: \"3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:15.994602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:15.994534    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnm2v\" (UniqueName: \"kubernetes.io/projected/3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08-kube-api-access-tnm2v\") pod \"dns-operator-controller-manager-8557cbdb84-ppnqs\" (UID: \"3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:16.063761 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:16.063717    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:16.349171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:16.349144    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"]
Oct 31 00:38:16.353226 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:16.353195    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9dfbaa_8253_40c8_ad31_c4fd7ea6fc08.slice/crio-2f1b90aa4aa0cd96ceb82fca88e02a08334ac723d101ea7357304c0927a92bc2 WatchSource:0}: Error finding container 2f1b90aa4aa0cd96ceb82fca88e02a08334ac723d101ea7357304c0927a92bc2: Status 404 returned error can't find the container with id 2f1b90aa4aa0cd96ceb82fca88e02a08334ac723d101ea7357304c0927a92bc2
Oct 31 00:38:16.442013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:16.441972    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs" event={"ID":"3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08","Type":"ContainerStarted","Data":"2f1b90aa4aa0cd96ceb82fca88e02a08334ac723d101ea7357304c0927a92bc2"}
Oct 31 00:38:18.456112 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:18.456024    4924 generic.go:356] "Generic (PLEG): container finished" podID="c25d6a67-6e6f-4703-8bd4-4374035bffbf" containerID="f85f4220e4b95915d57f02c4a660690f8848ed2cbc36927b7e8241e1a05ff640" exitCode=0
Oct 31 00:38:18.456561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:18.456108    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn" event={"ID":"c25d6a67-6e6f-4703-8bd4-4374035bffbf","Type":"ContainerDied","Data":"f85f4220e4b95915d57f02c4a660690f8848ed2cbc36927b7e8241e1a05ff640"}
Oct 31 00:38:19.207702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.207623    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"]
Oct 31 00:38:19.213387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.213354    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:19.221536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.221503    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"]
Oct 31 00:38:19.223325 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.223294    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-z5sbq\""
Oct 31 00:38:19.315693 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.315653    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8mwm\" (UniqueName: \"kubernetes.io/projected/48905ee6-4e74-4eb8-b8f8-061f475facdf-kube-api-access-l8mwm\") pod \"limitador-operator-controller-manager-5778bb4449-hgr7k\" (UID: \"48905ee6-4e74-4eb8-b8f8-061f475facdf\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:19.416631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.416557    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8mwm\" (UniqueName: \"kubernetes.io/projected/48905ee6-4e74-4eb8-b8f8-061f475facdf-kube-api-access-l8mwm\") pod \"limitador-operator-controller-manager-5778bb4449-hgr7k\" (UID: \"48905ee6-4e74-4eb8-b8f8-061f475facdf\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:19.435270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.435205    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8mwm\" (UniqueName: \"kubernetes.io/projected/48905ee6-4e74-4eb8-b8f8-061f475facdf-kube-api-access-l8mwm\") pod \"limitador-operator-controller-manager-5778bb4449-hgr7k\" (UID: \"48905ee6-4e74-4eb8-b8f8-061f475facdf\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:19.471328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.471239    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn" event={"ID":"c25d6a67-6e6f-4703-8bd4-4374035bffbf","Type":"ContainerStarted","Data":"e679fc2e3d771306b4dc2d5beac84090b7aec12d89421a67588246bfffe69762"}
Oct 31 00:38:19.472187 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.472086    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:19.509765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.509682    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn" podStartSLOduration=2.07547133 podStartE2EDuration="6.50966375s" totalImagesPullingTime="3.871066339s" totalInitContainerRuntime="563.126081ms" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:19.50786489 +0000 UTC m=+299.878648886" watchObservedRunningTime="2026-10-31 00:38:19.50966375 +0000 UTC m=+299.880447539"
Oct 31 00:38:19.531818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.531786    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:19.807851 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.807702    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:38:19.809930 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.809898    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:38:19.825735 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.825711    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:38:19.826261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.826244    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:38:19.854044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:19.854015    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"]
Oct 31 00:38:20.478993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:20.478955    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k" event={"ID":"48905ee6-4e74-4eb8-b8f8-061f475facdf","Type":"ContainerStarted","Data":"c1841579bde0b910d3ec35f79acaedce534cc27a20647d4504495d02e00cc187"}
Oct 31 00:38:20.865047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:20.865019    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 31 00:38:21.486938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:21.486899    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs" event={"ID":"3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08","Type":"ContainerStarted","Data":"e9926ed24dcb0a45abf5ced2cdb276f763100ed27cb406ee6b76baf6ea658e82"}
Oct 31 00:38:21.487396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:21.487132    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:21.512637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:21.512575    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs" podStartSLOduration=2.005604632 podStartE2EDuration="6.512556177s" totalImagesPullingTime="4.506951545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:21.509789557 +0000 UTC m=+301.880573348" watchObservedRunningTime="2026-10-31 00:38:21.512556177 +0000 UTC m=+301.883339967"
Oct 31 00:38:22.493454 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:22.493404    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb" event={"ID":"6971fbe7-353d-47dd-8dec-cefb775182d6","Type":"ContainerStarted","Data":"22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0"}
Oct 31 00:38:22.494024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:22.493988    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:22.496138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:22.496112    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k" event={"ID":"48905ee6-4e74-4eb8-b8f8-061f475facdf","Type":"ContainerStarted","Data":"18385883cd7245bb6385f6f4c4c8bf4ecc68609c82c7e7b4f9efb33f8e02ee00"}
Oct 31 00:38:22.496280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:22.496237    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:22.542820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:22.542771    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb" podStartSLOduration=2.278866006 podStartE2EDuration="11.54275639s" totalImagesPullingTime="9.263890384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:22.540277129 +0000 UTC m=+302.911060918" watchObservedRunningTime="2026-10-31 00:38:22.54275639 +0000 UTC m=+302.913540170"
Oct 31 00:38:22.591567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:22.591514    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k" podStartSLOduration=1.456431812 podStartE2EDuration="3.591498009s" totalImagesPullingTime="2.135066197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:22.591035329 +0000 UTC m=+302.961819140" watchObservedRunningTime="2026-10-31 00:38:22.591498009 +0000 UTC m=+302.962281797"
Oct 31 00:38:24.781735 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:24.781699    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"]
Oct 31 00:38:24.839236 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:24.839197    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"]
Oct 31 00:38:24.839449 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:24.839325    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:24.851131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:24.851105    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hckln\""
Oct 31 00:38:24.970326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:24.970285    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2g7j\" (UniqueName: \"kubernetes.io/projected/0d165765-1777-4387-952b-1b568c10a0f7-kube-api-access-w2g7j\") pod \"authorino-operator-7b7bb4bc77-pn9q9\" (UID: \"0d165765-1777-4387-952b-1b568c10a0f7\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:25.071111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:25.071005    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2g7j\" (UniqueName: \"kubernetes.io/projected/0d165765-1777-4387-952b-1b568c10a0f7-kube-api-access-w2g7j\") pod \"authorino-operator-7b7bb4bc77-pn9q9\" (UID: \"0d165765-1777-4387-952b-1b568c10a0f7\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:25.078485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:25.078459    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2g7j\" (UniqueName: \"kubernetes.io/projected/0d165765-1777-4387-952b-1b568c10a0f7-kube-api-access-w2g7j\") pod \"authorino-operator-7b7bb4bc77-pn9q9\" (UID: \"0d165765-1777-4387-952b-1b568c10a0f7\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:25.166419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:25.166380    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:25.444204 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:25.444134    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"]
Oct 31 00:38:25.446796 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:25.446760    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d165765_1777_4387_952b_1b568c10a0f7.slice/crio-976d91d58fb75a21f709567a89ee6bb74da849a549dc62fec406806979e5f3cc WatchSource:0}: Error finding container 976d91d58fb75a21f709567a89ee6bb74da849a549dc62fec406806979e5f3cc: Status 404 returned error can't find the container with id 976d91d58fb75a21f709567a89ee6bb74da849a549dc62fec406806979e5f3cc
Oct 31 00:38:25.448994 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:25.448974    4924 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:38:25.512902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:25.512869    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9" event={"ID":"0d165765-1777-4387-952b-1b568c10a0f7","Type":"ContainerStarted","Data":"976d91d58fb75a21f709567a89ee6bb74da849a549dc62fec406806979e5f3cc"}
Oct 31 00:38:27.525436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:27.525391    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9" event={"ID":"0d165765-1777-4387-952b-1b568c10a0f7","Type":"ContainerStarted","Data":"20b9916438130cee8c102c9f2478a05680e2dd2cfe72f2c4aa1339d7082fdc86"}
Oct 31 00:38:27.526096 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:27.525464    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:27.559647 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:27.559590    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9" podStartSLOduration=1.976696675 podStartE2EDuration="3.55957554s" totalImagesPullingTime="1.582878865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:27.557005004 +0000 UTC m=+307.927788793" watchObservedRunningTime="2026-10-31 00:38:27.55957554 +0000 UTC m=+307.930359384"
Oct 31 00:38:27.851158 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:38:27.851077    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:38:31.489223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:31.489188    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-54459f848c-4w4wn"
Oct 31 00:38:33.504477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:33.504448    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-hgr7k"
Oct 31 00:38:33.504889 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:33.504517    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-ppnqs"
Oct 31 00:38:34.509384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:34.509355    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:34.893654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:34.893620    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"]
Oct 31 00:38:34.897084 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:34.897063    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:34.913039 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:34.913014    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"]
Oct 31 00:38:34.959084 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:34.959047    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-tgctq\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:34.959268 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:34.959113    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5mnt\" (UniqueName: \"kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt\") pod \"kuadrant-operator-controller-manager-678d87766f-tgctq\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.059633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.059596    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5mnt\" (UniqueName: \"kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt\") pod \"kuadrant-operator-controller-manager-678d87766f-tgctq\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.059851 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.059674    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-tgctq\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.060102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.060079    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-tgctq\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.071378 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.071351    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5mnt\" (UniqueName: \"kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt\") pod \"kuadrant-operator-controller-manager-678d87766f-tgctq\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.209736 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.209647    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.498139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.498113    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"]
Oct 31 00:38:35.500946 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:35.500916    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0671ec6a_6f7d_47e3_a253_2f7c53213996.slice/crio-18e6979e85cc154401292b5c011118d42e5db7677af43c3c9cc74e5e01dc4385 WatchSource:0}: Error finding container 18e6979e85cc154401292b5c011118d42e5db7677af43c3c9cc74e5e01dc4385: Status 404 returned error can't find the container with id 18e6979e85cc154401292b5c011118d42e5db7677af43c3c9cc74e5e01dc4385
Oct 31 00:38:35.568342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.568313    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq" event={"ID":"0671ec6a-6f7d-47e3-a253-2f7c53213996","Type":"ContainerStarted","Data":"10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d"}
Oct 31 00:38:35.568727 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.568367    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq" event={"ID":"0671ec6a-6f7d-47e3-a253-2f7c53213996","Type":"ContainerStarted","Data":"18e6979e85cc154401292b5c011118d42e5db7677af43c3c9cc74e5e01dc4385"}
Oct 31 00:38:35.568727 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.568392    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:35.604530 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:35.604478    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq" podStartSLOduration=1.604463158 podStartE2EDuration="1.604463158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:35.60230904 +0000 UTC m=+315.973092840" watchObservedRunningTime="2026-10-31 00:38:35.604463158 +0000 UTC m=+315.975246959"
Oct 31 00:38:38.533019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:38.532986    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-pn9q9"
Oct 31 00:38:44.068834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.068792    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"]
Oct 31 00:38:44.073122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.073098    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.078243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.078221    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-x6xqw\""
Oct 31 00:38:44.083451 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.083423    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"]
Oct 31 00:38:44.122875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.122839    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.122898    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.122920    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.123022    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2089a27d-9052-4ead-a597-b014675e2de3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.123052    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.123071    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2089a27d-9052-4ead-a597-b014675e2de3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.123166    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123278 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.123196    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2089a27d-9052-4ead-a597-b014675e2de3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.123278 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.123213    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mct2n\" (UniqueName: \"kubernetes.io/projected/2089a27d-9052-4ead-a597-b014675e2de3-kube-api-access-mct2n\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.224522 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224480    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.224702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224542    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2089a27d-9052-4ead-a597-b014675e2de3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.224702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224568    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mct2n\" (UniqueName: \"kubernetes.io/projected/2089a27d-9052-4ead-a597-b014675e2de3-kube-api-access-mct2n\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.224702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224615    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.224702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224674    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.224702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224699    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225028 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224736    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2089a27d-9052-4ead-a597-b014675e2de3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225028 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224791    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225028 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.224818    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2089a27d-9052-4ead-a597-b014675e2de3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225191 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.225158    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.225179    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225307 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.225253    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225307 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.225275    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.225479 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.225462    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2089a27d-9052-4ead-a597-b014675e2de3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.226859 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.226836    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2089a27d-9052-4ead-a597-b014675e2de3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.227162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.227142    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2089a27d-9052-4ead-a597-b014675e2de3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.234552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.234504    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2089a27d-9052-4ead-a597-b014675e2de3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.234671 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.234596    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mct2n\" (UniqueName: \"kubernetes.io/projected/2089a27d-9052-4ead-a597-b014675e2de3-kube-api-access-mct2n\") pod \"data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9\" (UID: \"2089a27d-9052-4ead-a597-b014675e2de3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.387906 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.387815    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:44.698110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.698079    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"]
Oct 31 00:38:44.699622 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:44.699592    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2089a27d_9052_4ead_a597_b014675e2de3.slice/crio-959faeaa68ccc2bd89b2f993866260138d16c5d547110655428741cc5d2dbb8b WatchSource:0}: Error finding container 959faeaa68ccc2bd89b2f993866260138d16c5d547110655428741cc5d2dbb8b: Status 404 returned error can't find the container with id 959faeaa68ccc2bd89b2f993866260138d16c5d547110655428741cc5d2dbb8b
Oct 31 00:38:44.702324 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.702288    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:38:44.702410 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.702359    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:38:44.702410 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:44.702388    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:38:45.623773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.623720    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9" event={"ID":"2089a27d-9052-4ead-a597-b014675e2de3","Type":"ContainerStarted","Data":"b1790d462a41a2cc27f56fc67707b71774f1719a63ff81068f1e3c72d5e2352b"}
Oct 31 00:38:45.624193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.623789    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9" event={"ID":"2089a27d-9052-4ead-a597-b014675e2de3","Type":"ContainerStarted","Data":"959faeaa68ccc2bd89b2f993866260138d16c5d547110655428741cc5d2dbb8b"}
Oct 31 00:38:45.633729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.633699    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ffkxc"]
Oct 31 00:38:45.637273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.637252    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:45.641392 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.641362    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-czhr7\""
Oct 31 00:38:45.652117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.652066    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ffkxc"]
Oct 31 00:38:45.654594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.654545    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9" podStartSLOduration=1.654528703 podStartE2EDuration="1.654528703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:45.650039539 +0000 UTC m=+326.020823328" watchObservedRunningTime="2026-10-31 00:38:45.654528703 +0000 UTC m=+326.025312493"
Oct 31 00:38:45.738444 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.738408    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc\") pod \"authorino-57f7f8659f-ffkxc\" (UID: \"1a369fcc-b492-4309-b53f-8b372811ee9a\") " pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:45.839349 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.839305    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc\") pod \"authorino-57f7f8659f-ffkxc\" (UID: \"1a369fcc-b492-4309-b53f-8b372811ee9a\") " pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:45.850527 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.850497    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc\") pod \"authorino-57f7f8659f-ffkxc\" (UID: \"1a369fcc-b492-4309-b53f-8b372811ee9a\") " pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:45.963597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:45.963296    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:46.238514 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.238440    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ffkxc"]
Oct 31 00:38:46.242282 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:38:46.242247    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a369fcc_b492_4309_b53f_8b372811ee9a.slice/crio-ec5305c2c17bb4decd9122eee64348dce7ff9cd8eaa7c0c09886be675047ad8c WatchSource:0}: Error finding container ec5305c2c17bb4decd9122eee64348dce7ff9cd8eaa7c0c09886be675047ad8c: Status 404 returned error can't find the container with id ec5305c2c17bb4decd9122eee64348dce7ff9cd8eaa7c0c09886be675047ad8c
Oct 31 00:38:46.388702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.388667    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:46.393330 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.393304    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:46.576730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.576691    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:38:46.631001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.630959    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ffkxc" event={"ID":"1a369fcc-b492-4309-b53f-8b372811ee9a","Type":"ContainerStarted","Data":"ec5305c2c17bb4decd9122eee64348dce7ff9cd8eaa7c0c09886be675047ad8c"}
Oct 31 00:38:46.631466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.631265    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:46.632624 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.632601    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9"
Oct 31 00:38:46.664546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.664513    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"]
Oct 31 00:38:46.664791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.664765    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb" podUID="6971fbe7-353d-47dd-8dec-cefb775182d6" containerName="manager" containerID="cri-o://22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0" gracePeriod=10
Oct 31 00:38:46.910022 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.909998    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:46.950076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.950045    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp\" (UniqueName: \"kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp\") pod \"6971fbe7-353d-47dd-8dec-cefb775182d6\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") "
Oct 31 00:38:46.950249 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.950092    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume\") pod \"6971fbe7-353d-47dd-8dec-cefb775182d6\" (UID: \"6971fbe7-353d-47dd-8dec-cefb775182d6\") "
Oct 31 00:38:46.950728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.950701    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume" pod "6971fbe7-353d-47dd-8dec-cefb775182d6" (UID: "6971fbe7-353d-47dd-8dec-cefb775182d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:38:46.952203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:46.952183    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp" pod "6971fbe7-353d-47dd-8dec-cefb775182d6" (UID: "6971fbe7-353d-47dd-8dec-cefb775182d6"). InnerVolumeSpecName "kube-api-access-msxsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:47.051672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.051632    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msxsp\" (UniqueName: \"kubernetes.io/projected/6971fbe7-353d-47dd-8dec-cefb775182d6-kube-api-access-msxsp\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:47.051672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.051665    4924 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6971fbe7-353d-47dd-8dec-cefb775182d6-extensions-socket-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:47.637306 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.637273    4924 generic.go:356] "Generic (PLEG): container finished" podID="6971fbe7-353d-47dd-8dec-cefb775182d6" containerID="22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0" exitCode=0
Oct 31 00:38:47.637799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.637317    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"
Oct 31 00:38:47.637799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.637357    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb" event={"ID":"6971fbe7-353d-47dd-8dec-cefb775182d6","Type":"ContainerDied","Data":"22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0"}
Oct 31 00:38:47.637799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.637408    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb" event={"ID":"6971fbe7-353d-47dd-8dec-cefb775182d6","Type":"ContainerDied","Data":"b8bb4a3890292f8e7d68b0f0e61b465d5dc0e02d74955a50ea058f0c6dc9258c"}
Oct 31 00:38:47.637799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.637428    4924 scope.go:122] "RemoveContainer" containerID="22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0"
Oct 31 00:38:47.646452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.646438    4924 scope.go:122] "RemoveContainer" containerID="22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0"
Oct 31 00:38:47.646709 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:38:47.646690    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0\": container with ID starting with 22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0 not found: ID does not exist" containerID="22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0"
Oct 31 00:38:47.646815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.646717    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0"} err="failed to get container status \"22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0\": rpc error: code = NotFound desc = could not find container \"22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0\": container with ID starting with 22e9448d27944eefb66f0f9cff4b4675dc14280e34a79029f99ca8d2ff378fc0 not found: ID does not exist"
Oct 31 00:38:47.662347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.662324    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"]
Oct 31 00:38:47.666531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.666510    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-2qnmb"]
Oct 31 00:38:47.762509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:47.762479    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6971fbe7-353d-47dd-8dec-cefb775182d6" path="/var/lib/kubelet/pods/6971fbe7-353d-47dd-8dec-cefb775182d6/volumes"
Oct 31 00:38:50.366525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:50.366486    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ffkxc"]
Oct 31 00:38:50.656420 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:50.656326    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ffkxc" event={"ID":"1a369fcc-b492-4309-b53f-8b372811ee9a","Type":"ContainerStarted","Data":"0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3"}
Oct 31 00:38:50.672081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:50.672034    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-ffkxc" podStartSLOduration=1.623219309 podStartE2EDuration="5.67202133s" totalImagesPullingTime="4.048802021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:50.671023392 +0000 UTC m=+331.041807180" watchObservedRunningTime="2026-10-31 00:38:50.67202133 +0000 UTC m=+331.042805118"
Oct 31 00:38:51.661264 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:51.661208    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-ffkxc" podUID="1a369fcc-b492-4309-b53f-8b372811ee9a" containerName="authorino" containerID="cri-o://0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3" gracePeriod=30
Oct 31 00:38:51.943288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:51.943262    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:51.996789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:51.996731    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc\") pod \"1a369fcc-b492-4309-b53f-8b372811ee9a\" (UID: \"1a369fcc-b492-4309-b53f-8b372811ee9a\") "
Oct 31 00:38:51.998836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:51.998810    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc" pod "1a369fcc-b492-4309-b53f-8b372811ee9a" (UID: "1a369fcc-b492-4309-b53f-8b372811ee9a"). InnerVolumeSpecName "kube-api-access-pl9qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:38:52.097548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.097509    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/1a369fcc-b492-4309-b53f-8b372811ee9a-kube-api-access-pl9qc\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:38:52.667992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.667958    4924 generic.go:356] "Generic (PLEG): container finished" podID="1a369fcc-b492-4309-b53f-8b372811ee9a" containerID="0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3" exitCode=0
Oct 31 00:38:52.668483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.668007    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ffkxc"
Oct 31 00:38:52.668483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.668043    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ffkxc" event={"ID":"1a369fcc-b492-4309-b53f-8b372811ee9a","Type":"ContainerDied","Data":"0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3"}
Oct 31 00:38:52.668483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.668093    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ffkxc" event={"ID":"1a369fcc-b492-4309-b53f-8b372811ee9a","Type":"ContainerDied","Data":"ec5305c2c17bb4decd9122eee64348dce7ff9cd8eaa7c0c09886be675047ad8c"}
Oct 31 00:38:52.668483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.668117    4924 scope.go:122] "RemoveContainer" containerID="0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3"
Oct 31 00:38:52.679982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.679964    4924 scope.go:122] "RemoveContainer" containerID="0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3"
Oct 31 00:38:52.680307 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:38:52.680286    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3\": container with ID starting with 0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3 not found: ID does not exist" containerID="0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3"
Oct 31 00:38:52.680361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.680318    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3"} err="failed to get container status \"0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3\": rpc error: code = NotFound desc = could not find container \"0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3\": container with ID starting with 0ba178f2557541d2288015375159aa0d42f48bf2d527c657be37ffb82ceba7f3 not found: ID does not exist"
Oct 31 00:38:52.692152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.692124    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ffkxc"]
Oct 31 00:38:52.695991 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:52.695968    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ffkxc"]
Oct 31 00:38:53.762264 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:38:53.762230    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a369fcc-b492-4309-b53f-8b372811ee9a" path="/var/lib/kubelet/pods/1a369fcc-b492-4309-b53f-8b372811ee9a/volumes"
Oct 31 00:38:57.851396 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:38:57.851360    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:39:12.062709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.062670    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"]
Oct 31 00:39:12.063101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.063021    4924 cpu_manager.go:414] "containerMap: removing container" podUID="1a369fcc-b492-4309-b53f-8b372811ee9a" containerName="authorino"
Oct 31 00:39:12.063101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.063032    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a369fcc-b492-4309-b53f-8b372811ee9a" containerName="authorino"
Oct 31 00:39:12.063101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.063068    4924 cpu_manager.go:414] "containerMap: removing container" podUID="6971fbe7-353d-47dd-8dec-cefb775182d6" containerName="manager"
Oct 31 00:39:12.063101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.063073    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6971fbe7-353d-47dd-8dec-cefb775182d6" containerName="manager"
Oct 31 00:39:12.063230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.063131    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="6971fbe7-353d-47dd-8dec-cefb775182d6" containerName="manager"
Oct 31 00:39:12.063230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.063139    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a369fcc-b492-4309-b53f-8b372811ee9a" containerName="authorino"
Oct 31 00:39:12.066248 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.066232    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.075294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.075272    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 00:39:12.075429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.075341    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 00:39:12.075699 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.075674    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"]
Oct 31 00:39:12.138162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.138129    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 31 00:39:12.138318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.138181    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-t5dfb\""
Oct 31 00:39:12.161555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.161526    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vlht\" (UniqueName: \"kubernetes.io/projected/d69c98ac-3b05-4310-b563-6822a1985b8a-kube-api-access-2vlht\") pod \"postgres-7cdbf6dc44-8gmlk\" (UID: \"d69c98ac-3b05-4310-b563-6822a1985b8a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.161641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.161582    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d69c98ac-3b05-4310-b563-6822a1985b8a-data\") pod \"postgres-7cdbf6dc44-8gmlk\" (UID: \"d69c98ac-3b05-4310-b563-6822a1985b8a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.262921 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.262878    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vlht\" (UniqueName: \"kubernetes.io/projected/d69c98ac-3b05-4310-b563-6822a1985b8a-kube-api-access-2vlht\") pod \"postgres-7cdbf6dc44-8gmlk\" (UID: \"d69c98ac-3b05-4310-b563-6822a1985b8a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.263104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.262936    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d69c98ac-3b05-4310-b563-6822a1985b8a-data\") pod \"postgres-7cdbf6dc44-8gmlk\" (UID: \"d69c98ac-3b05-4310-b563-6822a1985b8a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.263367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.263345    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d69c98ac-3b05-4310-b563-6822a1985b8a-data\") pod \"postgres-7cdbf6dc44-8gmlk\" (UID: \"d69c98ac-3b05-4310-b563-6822a1985b8a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.272300 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.272281    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vlht\" (UniqueName: \"kubernetes.io/projected/d69c98ac-3b05-4310-b563-6822a1985b8a-kube-api-access-2vlht\") pod \"postgres-7cdbf6dc44-8gmlk\" (UID: \"d69c98ac-3b05-4310-b563-6822a1985b8a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.380693 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.380593    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:12.652280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.652253    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"]
Oct 31 00:39:12.654083 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:39:12.654055    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69c98ac_3b05_4310_b563_6822a1985b8a.slice/crio-8e982c4764bbb8753e30b8e7efe39626713072f04f0547a35dff01c2d8e62cfb WatchSource:0}: Error finding container 8e982c4764bbb8753e30b8e7efe39626713072f04f0547a35dff01c2d8e62cfb: Status 404 returned error can't find the container with id 8e982c4764bbb8753e30b8e7efe39626713072f04f0547a35dff01c2d8e62cfb
Oct 31 00:39:12.767439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:12.767403    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk" event={"ID":"d69c98ac-3b05-4310-b563-6822a1985b8a","Type":"ContainerStarted","Data":"8e982c4764bbb8753e30b8e7efe39626713072f04f0547a35dff01c2d8e62cfb"}
Oct 31 00:39:18.677398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.677363    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"]
Oct 31 00:39:18.682399 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.682375    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.691006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.690977    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 31 00:39:18.691184 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.691097    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:39:18.691184 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.691163    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 31 00:39:18.691319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.691221    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 31 00:39:18.691319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.691227    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 31 00:39:18.691384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.691323    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-sb856\""
Oct 31 00:39:18.691848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.691828    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"]
Oct 31 00:39:18.818907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.818874    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r7kg\" (UniqueName: \"kubernetes.io/projected/db013b71-39dc-4d04-be7a-a4290d785365-kube-api-access-4r7kg\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.819104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.818927    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db013b71-39dc-4d04-be7a-a4290d785365-manager-config\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.819104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.819027    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db013b71-39dc-4d04-be7a-a4290d785365-cert\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.819226 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.819129    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db013b71-39dc-4d04-be7a-a4290d785365-metrics-certs\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.920103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.920064    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db013b71-39dc-4d04-be7a-a4290d785365-metrics-certs\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.920286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.920232    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r7kg\" (UniqueName: \"kubernetes.io/projected/db013b71-39dc-4d04-be7a-a4290d785365-kube-api-access-4r7kg\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.920286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.920262    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db013b71-39dc-4d04-be7a-a4290d785365-manager-config\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.920374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.920287    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db013b71-39dc-4d04-be7a-a4290d785365-cert\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.920979 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.920949    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db013b71-39dc-4d04-be7a-a4290d785365-manager-config\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.922608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.922585    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db013b71-39dc-4d04-be7a-a4290d785365-metrics-certs\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.922776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.922741    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db013b71-39dc-4d04-be7a-a4290d785365-cert\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.928597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.928527    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r7kg\" (UniqueName: \"kubernetes.io/projected/db013b71-39dc-4d04-be7a-a4290d785365-kube-api-access-4r7kg\") pod \"jobset-controller-manager-55ccbfd5b8-5glzc\" (UID: \"db013b71-39dc-4d04-be7a-a4290d785365\") " pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:18.996996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:18.996961    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:19.273007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:19.272978    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"]
Oct 31 00:39:19.275187 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:39:19.275154    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb013b71_39dc_4d04_be7a_a4290d785365.slice/crio-69aab70e1b5d9b88ba236f59ff21567bb70e58776c4182d7c50ce02020b35985 WatchSource:0}: Error finding container 69aab70e1b5d9b88ba236f59ff21567bb70e58776c4182d7c50ce02020b35985: Status 404 returned error can't find the container with id 69aab70e1b5d9b88ba236f59ff21567bb70e58776c4182d7c50ce02020b35985
Oct 31 00:39:19.800547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:19.800516    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc" event={"ID":"db013b71-39dc-4d04-be7a-a4290d785365","Type":"ContainerStarted","Data":"69aab70e1b5d9b88ba236f59ff21567bb70e58776c4182d7c50ce02020b35985"}
Oct 31 00:39:22.823040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:22.822989    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc" event={"ID":"db013b71-39dc-4d04-be7a-a4290d785365","Type":"ContainerStarted","Data":"ea8359768aaee922c8f44b7472b0be292251ec81d20902838856015c9728acc8"}
Oct 31 00:39:22.823443 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:22.823121    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:22.840069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:22.840015    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc" podStartSLOduration=1.477390397 podStartE2EDuration="4.839996308s" totalImagesPullingTime="3.362605911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:22.838619713 +0000 UTC m=+363.209403504" watchObservedRunningTime="2026-10-31 00:39:22.839996308 +0000 UTC m=+363.210780098"
Oct 31 00:39:24.537792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:24.537740    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 31 00:39:24.835822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:24.835733    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk" event={"ID":"d69c98ac-3b05-4310-b563-6822a1985b8a","Type":"ContainerStarted","Data":"aeea491a18d7b9124d643c1590a6f6e00f3b227c23452d78bc0b13ca085b3e44"}
Oct 31 00:39:24.835968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:24.835879    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:24.853447 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:24.853402    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk" podStartSLOduration=0.974296338 podStartE2EDuration="12.853389435s" totalImagesPullingTime="11.879093097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:24.850881367 +0000 UTC m=+365.221665156" watchObservedRunningTime="2026-10-31 00:39:24.853389435 +0000 UTC m=+365.224173276"
Oct 31 00:39:27.852254 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:39:27.852212    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:39:30.870469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:30.870441    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-8gmlk"
Oct 31 00:39:33.834186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:39:33.834152    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-55ccbfd5b8-5glzc"
Oct 31 00:39:57.851931 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:39:57.851897    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:40:25.304457 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:25.304415    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": read tcp 10.132.0.2:33886->10.132.0.32:15021: read: connection reset by peer" start-of-body=
Oct 31 00:40:25.304861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:25.304505    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": read tcp 10.132.0.2:33886->10.132.0.32:15021: read: connection reset by peer"
Oct 31 00:40:26.168342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.168308    4924 generic.go:356] "Generic (PLEG): container finished" podID="fc1366be-1d30-4c52-a330-4164ddae0391" containerID="35428c9a70e8aba513c035e4236e332f199b7fc157001cf6807b6e4fa2e826e9" exitCode=0
Oct 31 00:40:26.168518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.168377    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" event={"ID":"fc1366be-1d30-4c52-a330-4164ddae0391","Type":"ContainerDied","Data":"35428c9a70e8aba513c035e4236e332f199b7fc157001cf6807b6e4fa2e826e9"}
Oct 31 00:40:26.168767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.168741    4924 scope.go:122] "RemoveContainer" containerID="35428c9a70e8aba513c035e4236e332f199b7fc157001cf6807b6e4fa2e826e9"
Oct 31 00:40:26.169896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.169867    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:40:26.169958 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.169937    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:40:26.170001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.169969    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:40:26.570376 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.570339    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4xsxg"]
Oct 31 00:40:26.573971 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.573951    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.579574 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.579547    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-czhr7\""
Oct 31 00:40:26.580968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.580942    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 31 00:40:26.582843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.582823    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4xsxg"]
Oct 31 00:40:26.728308 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.728270    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2df98\" (UniqueName: \"kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98\") pod \"authorino-6889cbd8c4-4xsxg\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") " pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.728482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.728318    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert\") pod \"authorino-6889cbd8c4-4xsxg\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") " pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.829374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.829294    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2df98\" (UniqueName: \"kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98\") pod \"authorino-6889cbd8c4-4xsxg\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") " pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.829374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.829343    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert\") pod \"authorino-6889cbd8c4-4xsxg\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") " pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.831804 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.831784    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert\") pod \"authorino-6889cbd8c4-4xsxg\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") " pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.837227 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.837203    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4xsxg"]
Oct 31 00:40:26.837428 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:40:26.837411    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2df98], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-4xsxg" podUID="8a34fb2f-4204-4c51-8353-f8fcfbb75f6f"
Oct 31 00:40:26.839018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.838993    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2df98\" (UniqueName: \"kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98\") pod \"authorino-6889cbd8c4-4xsxg\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") " pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:26.865271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.865243    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67bdb6df44-xc5cn"]
Oct 31 00:40:26.868520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.868505    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:26.877774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:26.877728    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67bdb6df44-xc5cn"]
Oct 31 00:40:27.030676 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.030627    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert\") pod \"authorino-67bdb6df44-xc5cn\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") " pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.030676 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.030677    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt7mw\" (UniqueName: \"kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw\") pod \"authorino-67bdb6df44-xc5cn\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") " pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.131544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.131461    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt7mw\" (UniqueName: \"kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw\") pod \"authorino-67bdb6df44-xc5cn\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") " pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.131691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.131572    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert\") pod \"authorino-67bdb6df44-xc5cn\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") " pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.133987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.133965    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert\") pod \"authorino-67bdb6df44-xc5cn\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") " pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.139848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.139823    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt7mw\" (UniqueName: \"kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw\") pod \"authorino-67bdb6df44-xc5cn\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") " pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.179978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.179946    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:40:27.180172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.180145    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" event={"ID":"fc1366be-1d30-4c52-a330-4164ddae0391","Type":"ContainerStarted","Data":"98a2565c1b81d942e0a973f02bf9a50046ba602cef5ae7fadca8c2ac28450198"}
Oct 31 00:40:27.180221 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.180177    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:27.223837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.223809    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:27.333208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.333172    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert\" (UniqueName: \"kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert\") pod \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") "
Oct 31 00:40:27.333385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.333349    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98\" (UniqueName: \"kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98\") pod \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\" (UID: \"8a34fb2f-4204-4c51-8353-f8fcfbb75f6f\") "
Oct 31 00:40:27.335422 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.335396    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert" pod "8a34fb2f-4204-4c51-8353-f8fcfbb75f6f" (UID: "8a34fb2f-4204-4c51-8353-f8fcfbb75f6f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:27.335541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.335438    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98" pod "8a34fb2f-4204-4c51-8353-f8fcfbb75f6f" (UID: "8a34fb2f-4204-4c51-8353-f8fcfbb75f6f"). InnerVolumeSpecName "kube-api-access-2df98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:27.435021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.434936    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2df98\" (UniqueName: \"kubernetes.io/projected/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-kube-api-access-2df98\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:27.435021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.434965    4924 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f-tls-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:27.478649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.478623    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67bdb6df44-xc5cn"]
Oct 31 00:40:27.480493 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:40:27.480466    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbfd3e79_5119_44c8_931f_e8f1cf6b5c58.slice/crio-5ec6b6db032b1ef1a7b1cbba9522a2a86fff898dadd3d34f1f22fb5db17d80e5 WatchSource:0}: Error finding container 5ec6b6db032b1ef1a7b1cbba9522a2a86fff898dadd3d34f1f22fb5db17d80e5: Status 404 returned error can't find the container with id 5ec6b6db032b1ef1a7b1cbba9522a2a86fff898dadd3d34f1f22fb5db17d80e5
Oct 31 00:40:27.851552 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:40:27.851517    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:40:27.910571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.910137    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:40:27.910571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.910258    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 31 00:40:27.910571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:27.910303    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 31 00:40:28.187518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:28.187427    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bdb6df44-xc5cn" event={"ID":"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58","Type":"ContainerStarted","Data":"5ec6b6db032b1ef1a7b1cbba9522a2a86fff898dadd3d34f1f22fb5db17d80e5"}
Oct 31 00:40:28.187518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:28.187451    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4xsxg"
Oct 31 00:40:28.243620 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:28.243589    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4xsxg"]
Oct 31 00:40:28.251090 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:28.251064    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4xsxg"]
Oct 31 00:40:28.907557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:28.907111    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 31 00:40:28.907557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:28.907174    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 31 00:40:29.219895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:29.219788    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bdb6df44-xc5cn" event={"ID":"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58","Type":"ContainerStarted","Data":"a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64"}
Oct 31 00:40:29.245385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:29.245333    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-67bdb6df44-xc5cn" podStartSLOduration=2.377879589 podStartE2EDuration="3.245317922s" totalImagesPullingTime="867.438333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:40:29.242587771 +0000 UTC m=+429.613371559" watchObservedRunningTime="2026-10-31 00:40:29.245317922 +0000 UTC m=+429.616101709"
Oct 31 00:40:29.762182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:29.762147    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a34fb2f-4204-4c51-8353-f8fcfbb75f6f" path="/var/lib/kubelet/pods/8a34fb2f-4204-4c51-8353-f8fcfbb75f6f/volumes"
Oct 31 00:40:29.913771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:29.910198    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 31 00:40:29.913771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:29.910264    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 31 00:40:30.914772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:30.911194    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 31 00:40:30.914772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:30.911261    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 31 00:40:31.911049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:31.911006    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 31 00:40:31.911216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:31.911073    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 31 00:40:32.910979 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:32.910948    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:40:32.911427 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:32.911256    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:40:32.911882 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:40:32.911865    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj"
Oct 31 00:40:57.851002 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:40:57.850963    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:41:27.851066 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:41:27.851034    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:41:48.171479 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.171443    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"]
Oct 31 00:41:48.175523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.175500    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.183508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.183067    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"]
Oct 31 00:41:48.183598 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.183535    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 31 00:41:48.183705 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.183590    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 00:41:48.183705 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.183643    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 00:41:48.183829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.183704    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-66c4l\""
Oct 31 00:41:48.344122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.344088    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581a4f76-b7c1-4293-9b06-1079af9b9e3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.344122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.344126    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.344326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.344229    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.344326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.344268    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.344326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.344294    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.344423 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.344377    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrldq\" (UniqueName: \"kubernetes.io/projected/581a4f76-b7c1-4293-9b06-1079af9b9e3b-kube-api-access-mrldq\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445380 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445288    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581a4f76-b7c1-4293-9b06-1079af9b9e3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445380 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445329    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445380 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445379    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445398    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445416    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445441    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrldq\" (UniqueName: \"kubernetes.io/projected/581a4f76-b7c1-4293-9b06-1079af9b9e3b-kube-api-access-mrldq\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445954    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.445986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.445978    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.446136 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.446031    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.447660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.447635    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581a4f76-b7c1-4293-9b06-1079af9b9e3b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.447887 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.447870    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581a4f76-b7c1-4293-9b06-1079af9b9e3b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.453469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.453447    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrldq\" (UniqueName: \"kubernetes.io/projected/581a4f76-b7c1-4293-9b06-1079af9b9e3b-kube-api-access-mrldq\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t\" (UID: \"581a4f76-b7c1-4293-9b06-1079af9b9e3b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.488166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.488127    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:48.763940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:48.763914    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"]
Oct 31 00:41:48.765857 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:41:48.765818    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod581a4f76_b7c1_4293_9b06_1079af9b9e3b.slice/crio-20bf4d3e7658abf5af8025e3778817dbddd0e04943e64d3833753f8dac5b758c WatchSource:0}: Error finding container 20bf4d3e7658abf5af8025e3778817dbddd0e04943e64d3833753f8dac5b758c: Status 404 returned error can't find the container with id 20bf4d3e7658abf5af8025e3778817dbddd0e04943e64d3833753f8dac5b758c
Oct 31 00:41:49.658627 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:49.658582    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" event={"ID":"581a4f76-b7c1-4293-9b06-1079af9b9e3b","Type":"ContainerStarted","Data":"20bf4d3e7658abf5af8025e3778817dbddd0e04943e64d3833753f8dac5b758c"}
Oct 31 00:41:50.297658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.297620    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"]
Oct 31 00:41:50.303240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.303194    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.313407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.313375    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 31 00:41:50.316412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.316322    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"]
Oct 31 00:41:50.469510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.469470    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.469730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.469583    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.469730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.469689    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf46l\" (UniqueName: \"kubernetes.io/projected/42092e39-d9c6-47f8-b6dd-b4921bdc955b-kube-api-access-qf46l\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.469847 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.469733    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.469847 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.469828    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.469917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.469863    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42092e39-d9c6-47f8-b6dd-b4921bdc955b-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.571181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571091    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42092e39-d9c6-47f8-b6dd-b4921bdc955b-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.571181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571164    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.571413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571214    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.571413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571263    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf46l\" (UniqueName: \"kubernetes.io/projected/42092e39-d9c6-47f8-b6dd-b4921bdc955b-kube-api-access-qf46l\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.571413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571291    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.571413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571343    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.572069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.571782    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.572069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.572024    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.572337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.572311    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.574141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.574101    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42092e39-d9c6-47f8-b6dd-b4921bdc955b-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.574260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.574180    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42092e39-d9c6-47f8-b6dd-b4921bdc955b-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.581919 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.581891    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf46l\" (UniqueName: \"kubernetes.io/projected/42092e39-d9c6-47f8-b6dd-b4921bdc955b-kube-api-access-qf46l\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6qm4\" (UID: \"42092e39-d9c6-47f8-b6dd-b4921bdc955b\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.642225 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.642184    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:50.679430 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.679378    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" event={"ID":"581a4f76-b7c1-4293-9b06-1079af9b9e3b","Type":"ContainerStarted","Data":"273a4c92e7a3776d3e426f7f31fdf0dc9007db85f0cc1d1558ccda46877aa159"}
Oct 31 00:41:50.711658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.711600    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podStartSLOduration=1.193806024 podStartE2EDuration="2.711585769s" totalImagesPullingTime="1.517779745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:50.709488579 +0000 UTC m=+511.080272369" watchObservedRunningTime="2026-10-31 00:41:50.711585769 +0000 UTC m=+511.082369551"
Oct 31 00:41:50.949097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:50.949067    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"]
Oct 31 00:41:50.950715 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:41:50.950670    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42092e39_d9c6_47f8_b6dd_b4921bdc955b.slice/crio-ecbafca4de3c14991b4f62f681e091b7d683ae86dc5483d1f8f15d5d5a65103d WatchSource:0}: Error finding container ecbafca4de3c14991b4f62f681e091b7d683ae86dc5483d1f8f15d5d5a65103d: Status 404 returned error can't find the container with id ecbafca4de3c14991b4f62f681e091b7d683ae86dc5483d1f8f15d5d5a65103d
Oct 31 00:41:51.488359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:51.488324    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:51.687231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:51.687186    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4" event={"ID":"42092e39-d9c6-47f8-b6dd-b4921bdc955b","Type":"ContainerStarted","Data":"faa27b6ab61aa741eafc92fbad54a027fb01b45b6fe2564b131a59f8e8d2e42d"}
Oct 31 00:41:51.687623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:51.687253    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4" event={"ID":"42092e39-d9c6-47f8-b6dd-b4921bdc955b","Type":"ContainerStarted","Data":"ecbafca4de3c14991b4f62f681e091b7d683ae86dc5483d1f8f15d5d5a65103d"}
Oct 31 00:41:51.708271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:51.708218    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4" podStartSLOduration=1.416471074 podStartE2EDuration="1.708203894s" totalImagesPullingTime="291.73282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:51.705262046 +0000 UTC m=+512.076045834" watchObservedRunningTime="2026-10-31 00:41:51.708203894 +0000 UTC m=+512.078987682"
Oct 31 00:41:52.643126 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:52.643091    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:41:54.163724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.163688    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"]
Oct 31 00:41:54.167459 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.167432    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.171588 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.171558    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 31 00:41:54.177305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.177279    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"]
Oct 31 00:41:54.209841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.209787    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.210045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.209857    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9d788e-efd8-48d9-94db-cc87b84bb703-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.210045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.209938    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.210045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.210030    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.210239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.210070    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqqb2\" (UniqueName: \"kubernetes.io/projected/af9d788e-efd8-48d9-94db-cc87b84bb703-kube-api-access-hqqb2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.210239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.210191    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.311243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311204    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.311441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311265    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.311441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311300    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqqb2\" (UniqueName: \"kubernetes.io/projected/af9d788e-efd8-48d9-94db-cc87b84bb703-kube-api-access-hqqb2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.311441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311373    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.311441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311433    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.311687 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311472    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9d788e-efd8-48d9-94db-cc87b84bb703-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.312099 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.311771    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.312099 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.312058    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.312546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.312523    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.314317 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.314297    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9d788e-efd8-48d9-94db-cc87b84bb703-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.314597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.314576    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af9d788e-efd8-48d9-94db-cc87b84bb703-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.320184 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.320118    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqqb2\" (UniqueName: \"kubernetes.io/projected/af9d788e-efd8-48d9-94db-cc87b84bb703-kube-api-access-hqqb2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk\" (UID: \"af9d788e-efd8-48d9-94db-cc87b84bb703\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.485557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.485477    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:54.767724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:54.767698    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"]
Oct 31 00:41:54.770297 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:41:54.770270    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9d788e_efd8_48d9_94db_cc87b84bb703.slice/crio-64ec833d95fce9aea84aa45c5297ea5784f28a5f623fc17110265af9503439a2 WatchSource:0}: Error finding container 64ec833d95fce9aea84aa45c5297ea5784f28a5f623fc17110265af9503439a2: Status 404 returned error can't find the container with id 64ec833d95fce9aea84aa45c5297ea5784f28a5f623fc17110265af9503439a2
Oct 31 00:41:55.716197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:55.716160    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk" event={"ID":"af9d788e-efd8-48d9-94db-cc87b84bb703","Type":"ContainerStarted","Data":"6858cce5e5287a3f64d8417c94cfbc769c8506a64b0f3c24e2bde26d90f2c554"}
Oct 31 00:41:55.716612 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:55.716228    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk" event={"ID":"af9d788e-efd8-48d9-94db-cc87b84bb703","Type":"ContainerStarted","Data":"64ec833d95fce9aea84aa45c5297ea5784f28a5f623fc17110265af9503439a2"}
Oct 31 00:41:55.739896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:55.738232    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk" podStartSLOduration=1.453026353 podStartE2EDuration="1.738212784s" totalImagesPullingTime="285.186431ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:55.738151016 +0000 UTC m=+516.108934808" watchObservedRunningTime="2026-10-31 00:41:55.738212784 +0000 UTC m=+516.108996576"
Oct 31 00:41:56.486409 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:56.486369    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:41:57.851933 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:41:57.851896    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:41:57.942977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:57.942944    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"]
Oct 31 00:41:57.946928 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:57.946892    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:57.967945 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:57.967913    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"]
Oct 31 00:41:58.068708 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.068670    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.068911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.068715    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.068911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.068808    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.068911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.068888    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.069019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.068928    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.069065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.069042    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t45c\" (UniqueName: \"kubernetes.io/projected/d0017f59-4050-462e-b9c7-e45a6cf0562e-kube-api-access-5t45c\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.069110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.069095    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.069174 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.069161    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.069206 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.069187    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0017f59-4050-462e-b9c7-e45a6cf0562e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.169944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.169842    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.169944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.169885    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0017f59-4050-462e-b9c7-e45a6cf0562e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.169944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.169930    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.169952    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.169985    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170003    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170020    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170060    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t45c\" (UniqueName: \"kubernetes.io/projected/d0017f59-4050-462e-b9c7-e45a6cf0562e-kube-api-access-5t45c\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170096    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170315    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170689 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170658    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170778    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.170944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.170894    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.171083 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.171060    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0017f59-4050-462e-b9c7-e45a6cf0562e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.172486 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.172457    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.172694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.172678    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.178440 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.178415    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0017f59-4050-462e-b9c7-e45a6cf0562e-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.178561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.178540    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t45c\" (UniqueName: \"kubernetes.io/projected/d0017f59-4050-462e-b9c7-e45a6cf0562e-kube-api-access-5t45c\") pod \"maas-default-gateway-openshift-default-6f54779779-p6krc\" (UID: \"d0017f59-4050-462e-b9c7-e45a6cf0562e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.263466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.263431    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:58.488691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.488595    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:58.501957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.501926    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:58.550606 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.550579    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"]
Oct 31 00:41:58.552255 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:41:58.552215    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0017f59_4050_462e_b9c7_e45a6cf0562e.slice/crio-4d5165c7f5ea29f8201912c6c8f09c30b886e2796f1a1037b34fe03ef7b358ba WatchSource:0}: Error finding container 4d5165c7f5ea29f8201912c6c8f09c30b886e2796f1a1037b34fe03ef7b358ba: Status 404 returned error can't find the container with id 4d5165c7f5ea29f8201912c6c8f09c30b886e2796f1a1037b34fe03ef7b358ba
Oct 31 00:41:58.554578 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.554539    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:41:58.554662 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.554625    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:41:58.554703 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.554670    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:41:58.741370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.741271    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" event={"ID":"d0017f59-4050-462e-b9c7-e45a6cf0562e","Type":"ContainerStarted","Data":"f19b2ee6c3dc582e4599207acbcbfea52e4370b95fdef39f1d2ddda6539dc27e"}
Oct 31 00:41:58.741370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.741340    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" event={"ID":"d0017f59-4050-462e-b9c7-e45a6cf0562e","Type":"ContainerStarted","Data":"4d5165c7f5ea29f8201912c6c8f09c30b886e2796f1a1037b34fe03ef7b358ba"}
Oct 31 00:41:58.755981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.755950    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:41:58.761787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:58.760443    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podStartSLOduration=1.760426565 podStartE2EDuration="1.760426565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:58.759799365 +0000 UTC m=+519.130583153" watchObservedRunningTime="2026-10-31 00:41:58.760426565 +0000 UTC m=+519.131210353"
Oct 31 00:41:59.264271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:59.264227    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:41:59.265942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:59.265915    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:41:59.266064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:41:59.265961    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:00.264668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:00.264626    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:42:00.265170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:00.264702    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:00.643034 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:00.642987    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:42:00.655534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:00.655503    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:42:00.764618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:00.764589    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4"
Oct 31 00:42:01.264571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:01.264526    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:42:01.264726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:01.264600    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:02.264455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:02.264420    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:42:02.264638 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:02.264489    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:03.264491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:03.264450    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:42:03.264906 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:03.264507    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:04.264195 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:04.264159    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:42:04.264380 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:04.264228    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:04.486335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:04.486295    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:42:04.498882 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:04.498855    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:42:04.787536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:04.787505    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:42:05.264176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:05.264085    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:42:05.264176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:05.264152    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 00:42:06.670005 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:06.669951    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:42:06.670432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:06.670320    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:42:06.671035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:06.671017    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc"
Oct 31 00:42:10.241226 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.241189    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-759d7ff9c9-vf8p2"]
Oct 31 00:42:10.244995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.244974    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.253428 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.252925    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-759d7ff9c9-vf8p2"]
Oct 31 00:42:10.411261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.411208    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcbs7\" (UniqueName: \"kubernetes.io/projected/7f05576a-ca0d-4e6f-9f10-382a47312af2-kube-api-access-fcbs7\") pod \"authorino-759d7ff9c9-vf8p2\" (UID: \"7f05576a-ca0d-4e6f-9f10-382a47312af2\") " pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.411439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.411357    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f05576a-ca0d-4e6f-9f10-382a47312af2-tls-cert\") pod \"authorino-759d7ff9c9-vf8p2\" (UID: \"7f05576a-ca0d-4e6f-9f10-382a47312af2\") " pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.512942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.512844    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcbs7\" (UniqueName: \"kubernetes.io/projected/7f05576a-ca0d-4e6f-9f10-382a47312af2-kube-api-access-fcbs7\") pod \"authorino-759d7ff9c9-vf8p2\" (UID: \"7f05576a-ca0d-4e6f-9f10-382a47312af2\") " pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.512942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.512923    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f05576a-ca0d-4e6f-9f10-382a47312af2-tls-cert\") pod \"authorino-759d7ff9c9-vf8p2\" (UID: \"7f05576a-ca0d-4e6f-9f10-382a47312af2\") " pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.515493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.515473    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f05576a-ca0d-4e6f-9f10-382a47312af2-tls-cert\") pod \"authorino-759d7ff9c9-vf8p2\" (UID: \"7f05576a-ca0d-4e6f-9f10-382a47312af2\") " pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.520550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.520525    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcbs7\" (UniqueName: \"kubernetes.io/projected/7f05576a-ca0d-4e6f-9f10-382a47312af2-kube-api-access-fcbs7\") pod \"authorino-759d7ff9c9-vf8p2\" (UID: \"7f05576a-ca0d-4e6f-9f10-382a47312af2\") " pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.558547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.558516    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759d7ff9c9-vf8p2"
Oct 31 00:42:10.832632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:10.832552    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-759d7ff9c9-vf8p2"]
Oct 31 00:42:10.835671 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:42:10.835640    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f05576a_ca0d_4e6f_9f10_382a47312af2.slice/crio-e7a2843908dcf814d0d9520cc10cc4c8ea05157e5db2ed0d5da0830e7616d334 WatchSource:0}: Error finding container e7a2843908dcf814d0d9520cc10cc4c8ea05157e5db2ed0d5da0830e7616d334: Status 404 returned error can't find the container with id e7a2843908dcf814d0d9520cc10cc4c8ea05157e5db2ed0d5da0830e7616d334
Oct 31 00:42:11.817872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:11.817836    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-759d7ff9c9-vf8p2" event={"ID":"7f05576a-ca0d-4e6f-9f10-382a47312af2","Type":"ContainerStarted","Data":"5f8d4efce466214c96b3fafbed65e6e6f085080b726939ca514435d25c2a8a0c"}
Oct 31 00:42:11.817872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:11.817880    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-759d7ff9c9-vf8p2" event={"ID":"7f05576a-ca0d-4e6f-9f10-382a47312af2","Type":"ContainerStarted","Data":"e7a2843908dcf814d0d9520cc10cc4c8ea05157e5db2ed0d5da0830e7616d334"}
Oct 31 00:42:11.850107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:11.850052    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-759d7ff9c9-vf8p2" podStartSLOduration=1.027669936 podStartE2EDuration="1.850034841s" totalImagesPullingTime="822.364905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:42:11.847195171 +0000 UTC m=+532.217978959" watchObservedRunningTime="2026-10-31 00:42:11.850034841 +0000 UTC m=+532.220818664"
Oct 31 00:42:11.919938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:11.919847    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67bdb6df44-xc5cn"]
Oct 31 00:42:11.920092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:11.920043    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-67bdb6df44-xc5cn" podUID="fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" containerName="authorino" containerID="cri-o://a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64" gracePeriod=30
Oct 31 00:42:12.189428 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.189399    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:42:12.334186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.334149    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert\" (UniqueName: \"kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert\") pod \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") "
Oct 31 00:42:12.334377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.334315    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw\" (UniqueName: \"kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw\") pod \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\" (UID: \"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58\") "
Oct 31 00:42:12.336451 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.336418    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw" pod "fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" (UID: "fbfd3e79-5119-44c8-931f-e8f1cf6b5c58"). InnerVolumeSpecName "kube-api-access-qt7mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:42:12.348819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.348779    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert" pod "fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" (UID: "fbfd3e79-5119-44c8-931f-e8f1cf6b5c58"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:42:12.435106 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.435022    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt7mw\" (UniqueName: \"kubernetes.io/projected/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-kube-api-access-qt7mw\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:42:12.435106 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.435054    4924 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58-tls-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:42:12.826380 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.826345    4924 generic.go:356] "Generic (PLEG): container finished" podID="fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" containerID="a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64" exitCode=0
Oct 31 00:42:12.826894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.826403    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bdb6df44-xc5cn"
Oct 31 00:42:12.826894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.826431    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bdb6df44-xc5cn" event={"ID":"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58","Type":"ContainerDied","Data":"a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64"}
Oct 31 00:42:12.826894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.826489    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bdb6df44-xc5cn" event={"ID":"fbfd3e79-5119-44c8-931f-e8f1cf6b5c58","Type":"ContainerDied","Data":"5ec6b6db032b1ef1a7b1cbba9522a2a86fff898dadd3d34f1f22fb5db17d80e5"}
Oct 31 00:42:12.826894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.826514    4924 scope.go:122] "RemoveContainer" containerID="a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64"
Oct 31 00:42:12.841575 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.841516    4924 scope.go:122] "RemoveContainer" containerID="a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64"
Oct 31 00:42:12.842015 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:42:12.841994    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64\": container with ID starting with a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64 not found: ID does not exist" containerID="a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64"
Oct 31 00:42:12.842116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.842029    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64"} err="failed to get container status \"a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64\": rpc error: code = NotFound desc = could not find container \"a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64\": container with ID starting with a51ab83eb5ed1e774ee949f31e223585ae9a070b37ab90621c0caaa88dde4e64 not found: ID does not exist"
Oct 31 00:42:12.854472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.854439    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67bdb6df44-xc5cn"]
Oct 31 00:42:12.858336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:12.858301    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67bdb6df44-xc5cn"]
Oct 31 00:42:13.762412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:13.762375    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" path="/var/lib/kubelet/pods/fbfd3e79-5119-44c8-931f-e8f1cf6b5c58/volumes"
Oct 31 00:42:18.996412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:18.996361    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"]
Oct 31 00:42:18.997004 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:18.996985    4924 cpu_manager.go:414] "containerMap: removing container" podUID="fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" containerName="authorino"
Oct 31 00:42:18.997106 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:18.997007    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" containerName="authorino"
Oct 31 00:42:18.997171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:18.997133    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbfd3e79-5119-44c8-931f-e8f1cf6b5c58" containerName="authorino"
Oct 31 00:42:19.001239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.001217    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.010680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.010651    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"]
Oct 31 00:42:19.012601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.012578    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 31 00:42:19.013304 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.013284    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6m4jw\""
Oct 31 00:42:19.021776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.021731    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-njw55"]
Oct 31 00:42:19.025241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.025226    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.035834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.035809    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-njw55"]
Oct 31 00:42:19.100280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.100244    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7e12ef-96ee-46bf-8813-36d45d5e9166-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-njw55\" (UID: \"ac7e12ef-96ee-46bf-8813-36d45d5e9166\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.100550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.100520    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd774ff8-af40-4f55-ac52-cbc18365d9dd-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-fn4lv\" (UID: \"bd774ff8-af40-4f55-ac52-cbc18365d9dd\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.103524 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.103500    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdzx6\" (UniqueName: \"kubernetes.io/projected/ac7e12ef-96ee-46bf-8813-36d45d5e9166-kube-api-access-vdzx6\") pod \"payload-pre-processing-7b98bb678c-njw55\" (UID: \"ac7e12ef-96ee-46bf-8813-36d45d5e9166\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.103723 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.103698    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbnhh\" (UniqueName: \"kubernetes.io/projected/bd774ff8-af40-4f55-ac52-cbc18365d9dd-kube-api-access-nbnhh\") pod \"payload-processing-55fbbbf9dc-fn4lv\" (UID: \"bd774ff8-af40-4f55-ac52-cbc18365d9dd\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.205028 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.204976    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7e12ef-96ee-46bf-8813-36d45d5e9166-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-njw55\" (UID: \"ac7e12ef-96ee-46bf-8813-36d45d5e9166\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.205202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.205035    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd774ff8-af40-4f55-ac52-cbc18365d9dd-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-fn4lv\" (UID: \"bd774ff8-af40-4f55-ac52-cbc18365d9dd\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.205202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.205094    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdzx6\" (UniqueName: \"kubernetes.io/projected/ac7e12ef-96ee-46bf-8813-36d45d5e9166-kube-api-access-vdzx6\") pod \"payload-pre-processing-7b98bb678c-njw55\" (UID: \"ac7e12ef-96ee-46bf-8813-36d45d5e9166\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.205202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.205156    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbnhh\" (UniqueName: \"kubernetes.io/projected/bd774ff8-af40-4f55-ac52-cbc18365d9dd-kube-api-access-nbnhh\") pod \"payload-processing-55fbbbf9dc-fn4lv\" (UID: \"bd774ff8-af40-4f55-ac52-cbc18365d9dd\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.205889 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.205862    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd774ff8-af40-4f55-ac52-cbc18365d9dd-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-fn4lv\" (UID: \"bd774ff8-af40-4f55-ac52-cbc18365d9dd\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.206025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.206001    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7e12ef-96ee-46bf-8813-36d45d5e9166-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-njw55\" (UID: \"ac7e12ef-96ee-46bf-8813-36d45d5e9166\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.213610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.213580    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbnhh\" (UniqueName: \"kubernetes.io/projected/bd774ff8-af40-4f55-ac52-cbc18365d9dd-kube-api-access-nbnhh\") pod \"payload-processing-55fbbbf9dc-fn4lv\" (UID: \"bd774ff8-af40-4f55-ac52-cbc18365d9dd\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.213739 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.213583    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdzx6\" (UniqueName: \"kubernetes.io/projected/ac7e12ef-96ee-46bf-8813-36d45d5e9166-kube-api-access-vdzx6\") pod \"payload-pre-processing-7b98bb678c-njw55\" (UID: \"ac7e12ef-96ee-46bf-8813-36d45d5e9166\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.321624 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.321592    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:19.344236 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.344206    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:19.602858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.602828    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"]
Oct 31 00:42:19.604328 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:42:19.604285    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd774ff8_af40_4f55_ac52_cbc18365d9dd.slice/crio-84212c58cd8cc56fc8b61b0ab7fc9ba78d1fd1cf917228829c204c1d80bf188b WatchSource:0}: Error finding container 84212c58cd8cc56fc8b61b0ab7fc9ba78d1fd1cf917228829c204c1d80bf188b: Status 404 returned error can't find the container with id 84212c58cd8cc56fc8b61b0ab7fc9ba78d1fd1cf917228829c204c1d80bf188b
Oct 31 00:42:19.650705 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.650682    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-njw55"]
Oct 31 00:42:19.652075 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:42:19.652051    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7e12ef_96ee_46bf_8813_36d45d5e9166.slice/crio-06f77a2ea4528e8a1f3e507df550bebb99261afba09f77e36b5c55bd0384f15b WatchSource:0}: Error finding container 06f77a2ea4528e8a1f3e507df550bebb99261afba09f77e36b5c55bd0384f15b: Status 404 returned error can't find the container with id 06f77a2ea4528e8a1f3e507df550bebb99261afba09f77e36b5c55bd0384f15b
Oct 31 00:42:19.877662 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.877562    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv" event={"ID":"bd774ff8-af40-4f55-ac52-cbc18365d9dd","Type":"ContainerStarted","Data":"84212c58cd8cc56fc8b61b0ab7fc9ba78d1fd1cf917228829c204c1d80bf188b"}
Oct 31 00:42:19.879075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:19.879045    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55" event={"ID":"ac7e12ef-96ee-46bf-8813-36d45d5e9166","Type":"ContainerStarted","Data":"06f77a2ea4528e8a1f3e507df550bebb99261afba09f77e36b5c55bd0384f15b"}
Oct 31 00:42:22.902486 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:22.902435    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv" event={"ID":"bd774ff8-af40-4f55-ac52-cbc18365d9dd","Type":"ContainerStarted","Data":"09b4042da20671d833cb0d22e958ba23efe56113fe7c1cdcc96e28d5ad718591"}
Oct 31 00:42:22.902961 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:22.902622    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:22.904031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:22.904012    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55" event={"ID":"ac7e12ef-96ee-46bf-8813-36d45d5e9166","Type":"ContainerStarted","Data":"3e0d5bf539e0db65e295dd39d1c3e42d376070625d7ecaa95de074ecda7cd786"}
Oct 31 00:42:22.904084 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:22.904062    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:22.918547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:22.918501    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv" podStartSLOduration=2.286733805 podStartE2EDuration="4.918461018s" totalImagesPullingTime="2.631727213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:42:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:42:22.917880278 +0000 UTC m=+543.288664066" watchObservedRunningTime="2026-10-31 00:42:22.918461018 +0000 UTC m=+543.289244807"
Oct 31 00:42:22.935496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:22.935450    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55" podStartSLOduration=1.386069337 podStartE2EDuration="3.935437808s" totalImagesPullingTime="2.549368471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:42:22.932979957 +0000 UTC m=+543.303763746" watchObservedRunningTime="2026-10-31 00:42:22.935437808 +0000 UTC m=+543.306221656"
Oct 31 00:42:27.852449 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:42:27.852410    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:42:28.595541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.595503    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"]
Oct 31 00:42:28.601874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.601851    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.607390 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.607366    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"]
Oct 31 00:42:28.608027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.608004    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 31 00:42:28.608027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.608023    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 31 00:42:28.608217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.608055    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 00:42:28.608217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.608087    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 00:42:28.608217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.608151    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-22nln\""
Oct 31 00:42:28.690899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.690868    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a41765f7-69c2-45e7-bd3e-93bcc31908a1-metrics-certs\") pod \"ai-gateway-controller-5bb95b6d85-g6jkp\" (UID: \"a41765f7-69c2-45e7-bd3e-93bcc31908a1\") " pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.690899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.690904    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5v4l\" (UniqueName: \"kubernetes.io/projected/a41765f7-69c2-45e7-bd3e-93bcc31908a1-kube-api-access-h5v4l\") pod \"ai-gateway-controller-5bb95b6d85-g6jkp\" (UID: \"a41765f7-69c2-45e7-bd3e-93bcc31908a1\") " pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.792131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.792096    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a41765f7-69c2-45e7-bd3e-93bcc31908a1-metrics-certs\") pod \"ai-gateway-controller-5bb95b6d85-g6jkp\" (UID: \"a41765f7-69c2-45e7-bd3e-93bcc31908a1\") " pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.792297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.792185    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5v4l\" (UniqueName: \"kubernetes.io/projected/a41765f7-69c2-45e7-bd3e-93bcc31908a1-kube-api-access-h5v4l\") pod \"ai-gateway-controller-5bb95b6d85-g6jkp\" (UID: \"a41765f7-69c2-45e7-bd3e-93bcc31908a1\") " pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.794556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.794528    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a41765f7-69c2-45e7-bd3e-93bcc31908a1-metrics-certs\") pod \"ai-gateway-controller-5bb95b6d85-g6jkp\" (UID: \"a41765f7-69c2-45e7-bd3e-93bcc31908a1\") " pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.800763 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.800730    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5v4l\" (UniqueName: \"kubernetes.io/projected/a41765f7-69c2-45e7-bd3e-93bcc31908a1-kube-api-access-h5v4l\") pod \"ai-gateway-controller-5bb95b6d85-g6jkp\" (UID: \"a41765f7-69c2-45e7-bd3e-93bcc31908a1\") " pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:28.917937 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:28.917848    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:29.193535 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:29.193511    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"]
Oct 31 00:42:29.195305 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:42:29.195270    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda41765f7_69c2_45e7_bd3e_93bcc31908a1.slice/crio-5fc2896eda9332476bafd65a0b9426e50b29349faf29061b74b53ac56c41987d WatchSource:0}: Error finding container 5fc2896eda9332476bafd65a0b9426e50b29349faf29061b74b53ac56c41987d: Status 404 returned error can't find the container with id 5fc2896eda9332476bafd65a0b9426e50b29349faf29061b74b53ac56c41987d
Oct 31 00:42:29.946663 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:29.946619    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp" event={"ID":"a41765f7-69c2-45e7-bd3e-93bcc31908a1","Type":"ContainerStarted","Data":"5fc2896eda9332476bafd65a0b9426e50b29349faf29061b74b53ac56c41987d"}
Oct 31 00:42:31.959590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:31.959549    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp" event={"ID":"a41765f7-69c2-45e7-bd3e-93bcc31908a1","Type":"ContainerStarted","Data":"376daf7584626ce9f4643c4a781c15dad24ba9e3af3b72b626f6b1af613182bf"}
Oct 31 00:42:31.959994 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:31.959668    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:31.976967 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:31.976915    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp" podStartSLOduration=1.706811614 podStartE2EDuration="3.976901331s" totalImagesPullingTime="2.270089717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:42:31.976114263 +0000 UTC m=+552.346898053" watchObservedRunningTime="2026-10-31 00:42:31.976901331 +0000 UTC m=+552.347685148"
Oct 31 00:42:33.913118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:33.913087    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-fn4lv"
Oct 31 00:42:33.913580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:33.913189    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-njw55"
Oct 31 00:42:42.968121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:42:42.968092    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5bb95b6d85-g6jkp"
Oct 31 00:42:57.852367 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:42:57.852327    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:43:12.349346 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.349264    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"]
Oct 31 00:43:12.354077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.354052    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.369993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.369032    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"]
Oct 31 00:43:12.376890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.376860    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-47f370-openshift-default-dockercfg-bxkmf\""
Oct 31 00:43:12.398734 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.398686    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.398914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.398829    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9jdx\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.398914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.398875    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.398914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.398906    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.399078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.399053    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.399123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.399105    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.399201 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.399183    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.399235 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.399222    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.399353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.399333    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.452384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.452341    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"]
Oct 31 00:43:12.456059 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.456035    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.469452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.469425    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"]
Oct 31 00:43:12.477105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.477081    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-b433e6-openshift-default-dockercfg-72t9t\""
Oct 31 00:43:12.500199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500165    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500203    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.500483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500232    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500311    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.500483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500382    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500424    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500456    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.500483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500480    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzvxx\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.500836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500611    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500657    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500685    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.500836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500708    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.500836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500730    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.500836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500809    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500841    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500864    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9jdx\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500916    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500937    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500959    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.500966    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.501152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.501042    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.501511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.501258    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.501684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.501661    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.502997 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.502970    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.503311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.503294    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.520202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.520171    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9jdx\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.520400 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.520377    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.602240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602154    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602218    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602246    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602284    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602334    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602352    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzvxx\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602399    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602437    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.602786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602464    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.603004 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602957    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.603004 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.602971    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.603194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.603024    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.603194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.603070    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.603194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.603084    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.604838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.604817    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.605316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.605298    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.619916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.619884    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzvxx\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.620668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.620643    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.677077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.676788    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:12.728982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.728950    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"]
Oct 31 00:43:12.732915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.732894    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.741399 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.741373    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-1d4a7e-openshift-default-dockercfg-sn4k2\""
Oct 31 00:43:12.754594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.748216    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"]
Oct 31 00:43:12.775776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.773950    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:12.805218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805125    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805163    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805191    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805346    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805397    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805560 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805485    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805560 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805532    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805666 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805562    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.805666 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.805607    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfz5l\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.816142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.816091    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"]
Oct 31 00:43:12.821364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.821338    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.829828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.827318    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f99ac0-openshift-default-dockercfg-5jrvn\""
Oct 31 00:43:12.833841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.832829    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"]
Oct 31 00:43:12.906443 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906407    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.906628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906455    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.906628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906542    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.906628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906583    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.906628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906602    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.906876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906793    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.906876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906851    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.906957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906893    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.906957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906921    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.906957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.906951    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907083    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.907156 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907128    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfz5l\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907213 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907203    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907284    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907388 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907368    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907449    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.907531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907497    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncs9\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.907661 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907632    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.907738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907657    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907788    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907832    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.907928 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.907879    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:12.908193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.908169    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.909285 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.909263    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.909758 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.909731    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.914856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.914833    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.915464 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.915446    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfz5l\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:12.971249 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.971217    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"]
Oct 31 00:43:12.972691 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:12.972664    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721f7484_545b_4792_9f79_0dfa23a82962.slice/crio-8c35af4ec95d437016683493695520a052d1106809b7fbe425be4cc82ce1da31 WatchSource:0}: Error finding container 8c35af4ec95d437016683493695520a052d1106809b7fbe425be4cc82ce1da31: Status 404 returned error can't find the container with id 8c35af4ec95d437016683493695520a052d1106809b7fbe425be4cc82ce1da31
Oct 31 00:43:12.975236 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.975198    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:12.975312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.975291    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:12.975349 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:12.975336    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009346    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009426    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009459    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009505    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009551    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009649    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009700    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009740    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.009802    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lncs9\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.010532    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.010846    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.011100    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.011740    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.012053    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.015127    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.018976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.018126    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.029897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.028367    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncs9\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.032055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.032028    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.061098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.061060    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:13.105228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.104608    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"]
Oct 31 00:43:13.107699 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:13.107472    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b570d4_6c8e_4454_a84a_37c4c4118471.slice/crio-93c3e921973890ccfc599ed28844ef29a3f2a424e23892cf4038155f53bfcda4 WatchSource:0}: Error finding container 93c3e921973890ccfc599ed28844ef29a3f2a424e23892cf4038155f53bfcda4: Status 404 returned error can't find the container with id 93c3e921973890ccfc599ed28844ef29a3f2a424e23892cf4038155f53bfcda4
Oct 31 00:43:13.110203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.110165    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.110274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.110250    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.110318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.110290    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.147139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.147107    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:13.192406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.191433    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" event={"ID":"721f7484-545b-4792-9f79-0dfa23a82962","Type":"ContainerStarted","Data":"e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"}
Oct 31 00:43:13.192406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.191502    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" event={"ID":"721f7484-545b-4792-9f79-0dfa23a82962","Type":"ContainerStarted","Data":"8c35af4ec95d437016683493695520a052d1106809b7fbe425be4cc82ce1da31"}
Oct 31 00:43:13.195495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.195427    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" event={"ID":"a2b570d4-6c8e-4454-a84a-37c4c4118471","Type":"ContainerStarted","Data":"93c3e921973890ccfc599ed28844ef29a3f2a424e23892cf4038155f53bfcda4"}
Oct 31 00:43:13.292244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.292174    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podStartSLOduration=1.292154697 podStartE2EDuration="1.292154697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:13.288550279 +0000 UTC m=+593.659334072" watchObservedRunningTime="2026-10-31 00:43:13.292154697 +0000 UTC m=+593.662938488"
Oct 31 00:43:13.361377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.361347    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"]
Oct 31 00:43:13.362418 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:13.362390    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdebed2_b2b3_4856_9af3_13a739db3fdc.slice/crio-914b9789a2cab2db4bad8c5f0ee4b58bfcae0f3bbebed5fe7a1bbb2c2bc54091 WatchSource:0}: Error finding container 914b9789a2cab2db4bad8c5f0ee4b58bfcae0f3bbebed5fe7a1bbb2c2bc54091: Status 404 returned error can't find the container with id 914b9789a2cab2db4bad8c5f0ee4b58bfcae0f3bbebed5fe7a1bbb2c2bc54091
Oct 31 00:43:13.364982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.364942    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.365064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.365022    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.365105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.365082    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.467982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.467948    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"]
Oct 31 00:43:13.469058 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:13.469022    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4b1e70_835c_4f70_a519_46ff69973f9f.slice/crio-db53cf888fdd557952033378e6efc0cb5cae1860acb31d3959a103edac81e375 WatchSource:0}: Error finding container db53cf888fdd557952033378e6efc0cb5cae1860acb31d3959a103edac81e375: Status 404 returned error can't find the container with id db53cf888fdd557952033378e6efc0cb5cae1860acb31d3959a103edac81e375
Oct 31 00:43:13.471916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.471874    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.472120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.472091    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.472297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.472272    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:13.677436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:13.677350    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:14.205144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.205096    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" event={"ID":"6f4b1e70-835c-4f70-a519-46ff69973f9f","Type":"ContainerStarted","Data":"0eb53ad932d7c315d9c79ded36071c1131fb1a3ba0c4f4b4be4fc28f4d81e53c"}
Oct 31 00:43:14.205367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.205174    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" event={"ID":"6f4b1e70-835c-4f70-a519-46ff69973f9f","Type":"ContainerStarted","Data":"db53cf888fdd557952033378e6efc0cb5cae1860acb31d3959a103edac81e375"}
Oct 31 00:43:14.207091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.207061    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" event={"ID":"afdebed2-b2b3-4856-9af3-13a739db3fdc","Type":"ContainerStarted","Data":"aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"}
Oct 31 00:43:14.207234 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.207119    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" event={"ID":"afdebed2-b2b3-4856-9af3-13a739db3fdc","Type":"ContainerStarted","Data":"914b9789a2cab2db4bad8c5f0ee4b58bfcae0f3bbebed5fe7a1bbb2c2bc54091"}
Oct 31 00:43:14.208855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.208823    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" event={"ID":"a2b570d4-6c8e-4454-a84a-37c4c4118471","Type":"ContainerStarted","Data":"ce4b1a13c849fcc6eb2c8fe71cf1c7c698c9cfb3e806d5adb81960689a74d0f2"}
Oct 31 00:43:14.225551 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.225498    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podStartSLOduration=2.225482582 podStartE2EDuration="2.225482582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:14.224015645 +0000 UTC m=+594.594799448" watchObservedRunningTime="2026-10-31 00:43:14.225482582 +0000 UTC m=+594.596266372"
Oct 31 00:43:14.245228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.245177    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podStartSLOduration=2.24516373 podStartE2EDuration="2.24516373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:14.243036592 +0000 UTC m=+594.613820382" watchObservedRunningTime="2026-10-31 00:43:14.24516373 +0000 UTC m=+594.615947519"
Oct 31 00:43:14.260596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.260525    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podStartSLOduration=2.2605085799999998 podStartE2EDuration="2.26050858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:14.260187557 +0000 UTC m=+594.630971347" watchObservedRunningTime="2026-10-31 00:43:14.26050858 +0000 UTC m=+594.631292369"
Oct 31 00:43:14.683670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.683636    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:14.775809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.775727    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:14.781404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:14.781380    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:15.061965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.061927    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:15.067566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.067539    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:15.148137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.148090    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:15.153957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.153930    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:15.215362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.215269    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:15.215362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.215325    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:15.215669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.215643    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:15.215776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.215696    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:15.216402 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.216377    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:15.216513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.216435    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:15.216608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.216597    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:15.216654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:15.216634    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:17.364340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.364297    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"]
Oct 31 00:43:17.368611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.368583    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.381086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.381051    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"]
Oct 31 00:43:17.467531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.467499    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-47f370\""
Oct 31 00:43:17.467794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.467738    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw59w\" (UniqueName: \"kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w\") pod \"payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.467965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.467944    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.569552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.569514    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.569732 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.569600    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw59w\" (UniqueName: \"kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w\") pod \"payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.570270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.570249    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.579033 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.579007    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw59w\" (UniqueName: \"kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w\") pod \"payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.684565 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.684481    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:17.965017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:17.964991    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"]
Oct 31 00:43:17.966811 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:17.966779    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae8692f_accb_41df_8b30_6955de4d0e5e.slice/crio-a7b704b4cfedd2c34776d44216ffc91f24445fbf4ffa8b94b573c4493837fbdb WatchSource:0}: Error finding container a7b704b4cfedd2c34776d44216ffc91f24445fbf4ffa8b94b573c4493837fbdb: Status 404 returned error can't find the container with id a7b704b4cfedd2c34776d44216ffc91f24445fbf4ffa8b94b573c4493837fbdb
Oct 31 00:43:18.238663 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:18.238568    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" event={"ID":"3ae8692f-accb-41df-8b30-6955de4d0e5e","Type":"ContainerStarted","Data":"0a04fbe3117128c96f38713739570e3490e31a9791d896b12be6eac5a1c67714"}
Oct 31 00:43:18.238663 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:18.238625    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" event={"ID":"3ae8692f-accb-41df-8b30-6955de4d0e5e","Type":"ContainerStarted","Data":"a7b704b4cfedd2c34776d44216ffc91f24445fbf4ffa8b94b573c4493837fbdb"}
Oct 31 00:43:18.238868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:18.238768    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:18.255556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:18.255503    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" podStartSLOduration=1.255490625 podStartE2EDuration="1.255490625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:18.254513971 +0000 UTC m=+598.625297751" watchObservedRunningTime="2026-10-31 00:43:18.255490625 +0000 UTC m=+598.626274447"
Oct 31 00:43:19.046659 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.046621    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"]
Oct 31 00:43:19.050470 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.050448    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.057455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.057410    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-7f1496\""
Oct 31 00:43:19.060444 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.060416    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"]
Oct 31 00:43:19.093401 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.093358    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2phh\" (UniqueName: \"kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh\") pod \"payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.093583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.093424    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.208521 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.204607    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2phh\" (UniqueName: \"kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh\") pod \"payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.208521 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.204665    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.228773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.226171    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.248636 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.248598    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2phh\" (UniqueName: \"kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh\") pod \"payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.365437 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.365329    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:19.647619 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:19.647590    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54fe85ab_8589_424f_aea2_4b5fe867ca4c.slice/crio-793bc76e4fef21befa4e9563ed7721b9a1f58367ce5c6e9d89944665b7d89332 WatchSource:0}: Error finding container 793bc76e4fef21befa4e9563ed7721b9a1f58367ce5c6e9d89944665b7d89332: Status 404 returned error can't find the container with id 793bc76e4fef21befa4e9563ed7721b9a1f58367ce5c6e9d89944665b7d89332
Oct 31 00:43:19.647774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.647632    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"]
Oct 31 00:43:19.883584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.883550    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:43:19.886114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.886090    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:43:19.892118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.892098    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:43:19.895491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.895424    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:43:19.959860 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.959823    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"]
Oct 31 00:43:19.964794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.964767    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:19.976573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:19.975885    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"]
Oct 31 00:43:20.012989 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.012953    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xf5q\" (UniqueName: \"kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q\") pod \"payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.012989 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.012998    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.036506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.036475    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-f99ac0-dockercfg-nt8gv\""
Oct 31 00:43:20.036671 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.036553    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-f99ac0\""
Oct 31 00:43:20.115096 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.115042    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xf5q\" (UniqueName: \"kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q\") pod \"payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.132231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.115106    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.132231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.116179    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.154294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.153854    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xf5q\" (UniqueName: \"kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q\") pod \"payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.258103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.258061    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" event={"ID":"54fe85ab-8589-424f-aea2-4b5fe867ca4c","Type":"ContainerStarted","Data":"bb30d968d5bd5e02cbe1d3f6c6ce96a33498f77f26816a371347b06a97497487"}
Oct 31 00:43:20.258272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.258125    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" event={"ID":"54fe85ab-8589-424f-aea2-4b5fe867ca4c","Type":"ContainerStarted","Data":"793bc76e4fef21befa4e9563ed7721b9a1f58367ce5c6e9d89944665b7d89332"}
Oct 31 00:43:20.258272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.258156    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:20.277613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.277554    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" podStartSLOduration=1.277540942 podStartE2EDuration="1.277540942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:20.274181289 +0000 UTC m=+600.644965077" watchObservedRunningTime="2026-10-31 00:43:20.277540942 +0000 UTC m=+600.648324731"
Oct 31 00:43:20.285158 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.285131    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:20.571614 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.571586    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"]
Oct 31 00:43:20.573973 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:20.573944    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f27690_bb8e_4159_b918_84baff256f09.slice/crio-0ee13b04a36c3e93aaec6c0623e372cffabcff606f03a78f2bed121fe3cdeb9e WatchSource:0}: Error finding container 0ee13b04a36c3e93aaec6c0623e372cffabcff606f03a78f2bed121fe3cdeb9e: Status 404 returned error can't find the container with id 0ee13b04a36c3e93aaec6c0623e372cffabcff606f03a78f2bed121fe3cdeb9e
Oct 31 00:43:20.709250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.709211    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"]
Oct 31 00:43:20.713836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.713809    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:20.723190 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.722607    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"]
Oct 31 00:43:20.724797 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.724770    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-1d4a7e-dockercfg-g6fss\""
Oct 31 00:43:20.734489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.734464    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-1d4a7e\""
Oct 31 00:43:20.823262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.823175    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:20.823262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.823223    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zr5l\" (UniqueName: \"kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l\") pod \"payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:20.924312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.924274    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:20.924519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.924320    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zr5l\" (UniqueName: \"kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l\") pod \"payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:20.925101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.925075    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:20.933157 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:20.933127    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zr5l\" (UniqueName: \"kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l\") pod \"payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:21.030287 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.030242    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:21.265255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.265215    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" event={"ID":"e9f27690-bb8e-4159-b918-84baff256f09","Type":"ContainerStarted","Data":"486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7"}
Oct 31 00:43:21.265712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.265272    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" event={"ID":"e9f27690-bb8e-4159-b918-84baff256f09","Type":"ContainerStarted","Data":"0ee13b04a36c3e93aaec6c0623e372cffabcff606f03a78f2bed121fe3cdeb9e"}
Oct 31 00:43:21.265712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.265298    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:21.288191 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.288130    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" podStartSLOduration=2.288113217 podStartE2EDuration="2.288113217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:21.284648087 +0000 UTC m=+601.655431876" watchObservedRunningTime="2026-10-31 00:43:21.288113217 +0000 UTC m=+601.658897007"
Oct 31 00:43:21.310350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.309733    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"]
Oct 31 00:43:21.311697 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:21.311672    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32703c40_6595_4a66_acf4_e32a468b2c8b.slice/crio-11bd3902e3aa776aea5338a802ed23be977f733c69014b485e73862b6c988847 WatchSource:0}: Error finding container 11bd3902e3aa776aea5338a802ed23be977f733c69014b485e73862b6c988847: Status 404 returned error can't find the container with id 11bd3902e3aa776aea5338a802ed23be977f733c69014b485e73862b6c988847
Oct 31 00:43:21.331894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.331866    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"]
Oct 31 00:43:21.337872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.337806    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.347678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.347644    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"]
Oct 31 00:43:21.348008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.347970    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-b433e6\""
Oct 31 00:43:21.348145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.348087    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-b433e6-dockercfg-mbd8q\""
Oct 31 00:43:21.372347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.372318    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"]
Oct 31 00:43:21.377011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.376982    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.387992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.381600    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"]
Oct 31 00:43:21.433661 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.433076    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hpzl\" (UniqueName: \"kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl\") pod \"payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.433661 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.433144    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.433661 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.433249    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.433661 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.433367    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l8nm\" (UniqueName: \"kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm\") pod \"payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.543975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.536402    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.543975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.536539    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l8nm\" (UniqueName: \"kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm\") pod \"payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.543975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.536583    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hpzl\" (UniqueName: \"kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl\") pod \"payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.543975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.536624    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.543975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.537634    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.543975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.538415    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.565339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.564417    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l8nm\" (UniqueName: \"kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm\") pod \"payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.565339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.565059    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hpzl\" (UniqueName: \"kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl\") pod \"payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:21.700277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.698236    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:21.720664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:21.720578    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:22.038947 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.038910    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"]
Oct 31 00:43:22.051550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.051523    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"]
Oct 31 00:43:22.056209 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:22.056176    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5f27fd_5a43_49b3_8644_8096e11165ed.slice/crio-5d06e8b5543f7480bba8c76888dcc66418b2df95f8d42b05fe6332e7fa81b6f8 WatchSource:0}: Error finding container 5d06e8b5543f7480bba8c76888dcc66418b2df95f8d42b05fe6332e7fa81b6f8: Status 404 returned error can't find the container with id 5d06e8b5543f7480bba8c76888dcc66418b2df95f8d42b05fe6332e7fa81b6f8
Oct 31 00:43:22.277377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.277333    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" event={"ID":"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7","Type":"ContainerStarted","Data":"4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb"}
Oct 31 00:43:22.277865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.277396    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" event={"ID":"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7","Type":"ContainerStarted","Data":"cf0d520354aa20078f87d042eb45488f2bf68b30ee8b1d68d3b8b2ccb0e33063"}
Oct 31 00:43:22.277865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.277551    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:22.278938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.278911    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" event={"ID":"cb5f27fd-5a43-49b3-8644-8096e11165ed","Type":"ContainerStarted","Data":"59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121"}
Oct 31 00:43:22.279027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.278962    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" event={"ID":"cb5f27fd-5a43-49b3-8644-8096e11165ed","Type":"ContainerStarted","Data":"5d06e8b5543f7480bba8c76888dcc66418b2df95f8d42b05fe6332e7fa81b6f8"}
Oct 31 00:43:22.279027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.278993    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:22.280693 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.280672    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" event={"ID":"32703c40-6595-4a66-acf4-e32a468b2c8b","Type":"ContainerStarted","Data":"5a0df213f4dfa9769e395268de4c653aa91f056ff3336f7a2b435571d81a3d08"}
Oct 31 00:43:22.280793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.280718    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" event={"ID":"32703c40-6595-4a66-acf4-e32a468b2c8b","Type":"ContainerStarted","Data":"11bd3902e3aa776aea5338a802ed23be977f733c69014b485e73862b6c988847"}
Oct 31 00:43:22.281056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.281042    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:22.376358 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.376255    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" podStartSLOduration=1.37624061 podStartE2EDuration="1.37624061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:22.322171332 +0000 UTC m=+602.692955121" watchObservedRunningTime="2026-10-31 00:43:22.37624061 +0000 UTC m=+602.747024398"
Oct 31 00:43:22.378072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.378006    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" podStartSLOduration=2.377993086 podStartE2EDuration="2.377993086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:22.375508774 +0000 UTC m=+602.746292564" watchObservedRunningTime="2026-10-31 00:43:22.377993086 +0000 UTC m=+602.748776874"
Oct 31 00:43:22.422196 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:22.422125    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" podStartSLOduration=1.422105189 podStartE2EDuration="1.422105189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:22.418369682 +0000 UTC m=+602.789153471" watchObservedRunningTime="2026-10-31 00:43:22.422105189 +0000 UTC m=+602.792888978"
Oct 31 00:43:27.862412 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:43:27.862377    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:43:29.254820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:29.254787    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:43:31.259966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:31.259924    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:43:31.260375 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:31.260001    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:43:31.269352 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:31.269323    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:43:32.283737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:32.283698    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:43:33.290707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:33.290678    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:43:33.292666 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:33.292645    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:43:34.302015 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.301238    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:43:34.349723 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.349681    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"]
Oct 31 00:43:34.354796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.354771    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.363934 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.363898    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5c2a03\"/\"simulator-w4-5c2a03-kserve-self-signed-certs\""
Oct 31 00:43:34.364680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.364658    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5c2a03\"/\"default-dockercfg-6cdqc\""
Oct 31 00:43:34.375224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.375190    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"]
Oct 31 00:43:34.390812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.390778    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-5c2a03\"/\"kube-root-ca.crt\""
Oct 31 00:43:34.390979 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.390837    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-5c2a03\"/\"openshift-service-ca.crt\""
Oct 31 00:43:34.482957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.482920    4924 generic.go:356] "Generic (PLEG): container finished" podID="721f7484-545b-4792-9f79-0dfa23a82962" containerID="e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9" exitCode=0
Oct 31 00:43:34.483142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.482995    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" event={"ID":"721f7484-545b-4792-9f79-0dfa23a82962","Type":"ContainerDied","Data":"e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"}
Oct 31 00:43:34.483444 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.483414    4924 scope.go:122] "RemoveContainer" containerID="e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"
Oct 31 00:43:34.484766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.484723    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:34.484832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.484809    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:34.484872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.484844    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:34.490618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.490534    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.490618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.490591    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.490826 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.490804    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.490896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.490855    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsg6v\" (UniqueName: \"kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.491529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.491346    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.491529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.491412    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.598101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.593010    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.598101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.593066    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.598101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.593095    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.598101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.593135    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsg6v\" (UniqueName: \"kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.598101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.593234    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.598101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.593273    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.610374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.609807    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.631245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.631186    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.631768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.631699    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.632510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.632466    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.633655 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.633602    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.650443 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.649677    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsg6v\" (UniqueName: \"kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v\") pod \"simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") " pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:34.675837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:34.675297    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:35.074123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.072416    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"]
Oct 31 00:43:35.076819 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:35.076788    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac16d450_266c_4cce_a250_0725443133c4.slice/crio-8272c878f6e309d4c46cc8567834684062754aedd9973ecfa9198db4b48b1361 WatchSource:0}: Error finding container 8272c878f6e309d4c46cc8567834684062754aedd9973ecfa9198db4b48b1361: Status 404 returned error can't find the container with id 8272c878f6e309d4c46cc8567834684062754aedd9973ecfa9198db4b48b1361
Oct 31 00:43:35.496822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.493782    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" event={"ID":"ac16d450-266c-4cce-a250-0725443133c4","Type":"ContainerStarted","Data":"cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389"}
Oct 31 00:43:35.496822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.493836    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" event={"ID":"ac16d450-266c-4cce-a250-0725443133c4","Type":"ContainerStarted","Data":"8272c878f6e309d4c46cc8567834684062754aedd9973ecfa9198db4b48b1361"}
Oct 31 00:43:35.510167 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.510114    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" event={"ID":"721f7484-545b-4792-9f79-0dfa23a82962","Type":"ContainerStarted","Data":"9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f"}
Oct 31 00:43:35.556817 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.551894    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podStartSLOduration=1.5518725039999999 podStartE2EDuration="1.551872504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:35.537177675 +0000 UTC m=+615.907961466" watchObservedRunningTime="2026-10-31 00:43:35.551872504 +0000 UTC m=+615.922656294"
Oct 31 00:43:35.676166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.676097    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:35.677251 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.677194    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:35.677513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.677496    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 00:43:35.677635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:35.677618    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 00:43:36.677796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:36.677522    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 00:43:36.677796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:36.677574    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 00:43:37.678397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:37.677952    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 00:43:37.678397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:37.678019    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 00:43:38.677968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:38.677456    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 00:43:38.677968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:38.677542    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 00:43:39.680776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:39.677519    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 00:43:39.680776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:39.677586    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 00:43:40.681393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:40.678126    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 00:43:40.681393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:40.678191    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 00:43:41.649859 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.649197    4924 generic.go:356] "Generic (PLEG): container finished" podID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerID="aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f" exitCode=0
Oct 31 00:43:41.649859 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.649449    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" event={"ID":"afdebed2-b2b3-4856-9af3-13a739db3fdc","Type":"ContainerDied","Data":"aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"}
Oct 31 00:43:41.652736 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.652362    4924 scope.go:122] "RemoveContainer" containerID="aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"
Oct 31 00:43:41.655955 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.655323    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:41.655955 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.655423    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:41.655955 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.655535    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:41.660354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.658047    4924 generic.go:356] "Generic (PLEG): container finished" podID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerID="0eb53ad932d7c315d9c79ded36071c1131fb1a3ba0c4f4b4be4fc28f4d81e53c" exitCode=0
Oct 31 00:43:41.660354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.658109    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" event={"ID":"6f4b1e70-835c-4f70-a519-46ff69973f9f","Type":"ContainerDied","Data":"0eb53ad932d7c315d9c79ded36071c1131fb1a3ba0c4f4b4be4fc28f4d81e53c"}
Oct 31 00:43:41.660354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.658544    4924 scope.go:122] "RemoveContainer" containerID="0eb53ad932d7c315d9c79ded36071c1131fb1a3ba0c4f4b4be4fc28f4d81e53c"
Oct 31 00:43:41.663221 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.663163    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:41.663668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.663596    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:41.663976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.663652    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:41.690042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.689701    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:41.693016 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.692128    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:41.693016 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:41.692968    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:43:42.668406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:42.668368    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" event={"ID":"afdebed2-b2b3-4856-9af3-13a739db3fdc","Type":"ContainerStarted","Data":"8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf"}
Oct 31 00:43:42.670820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:42.670792    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" event={"ID":"6f4b1e70-835c-4f70-a519-46ff69973f9f","Type":"ContainerStarted","Data":"4f3bd7c408cc401d8478de0c7fbfc58780e1e3b87b2a7243f08d5e82bd54f2f5"}
Oct 31 00:43:43.061952 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:43.061916    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:43.063195 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:43.063165    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 31 00:43:43.063325 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:43.063225    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 31 00:43:43.152385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:43.152008    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:43.153581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:43.153552    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 00:43:43.153814 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:43.153787    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 00:43:44.061452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.061412    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 31 00:43:44.061633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.061470    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 31 00:43:44.149770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.149177    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 00:43:44.149770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.149243    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 00:43:44.677543 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.676275    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"]
Oct 31 00:43:44.687441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.686894    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"]
Oct 31 00:43:44.687441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.686944    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:44.687771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.687722    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.772044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.772016    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:44.773283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.773238    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-1d4a7e\"/\"kube-root-ca.crt\""
Oct 31 00:43:44.773434 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.773378    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d4a7e\"/\"default-dockercfg-f794t\""
Oct 31 00:43:44.773505 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.773458    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d4a7e\"/\"simulator-w3-1d4a7e-kserve-self-signed-certs\""
Oct 31 00:43:44.773671 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.773630    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-1d4a7e\"/\"openshift-service-ca.crt\""
Oct 31 00:43:44.783513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.783391    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.784018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.783849    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.785052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.783884    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzcqs\" (UniqueName: \"kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.785052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.784285    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.785052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.784344    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.785052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.784405    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.885526 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.885476    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.885688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.885533    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzcqs\" (UniqueName: \"kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.885688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.885642    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.885811 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.885791    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.885902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.885885    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.885963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.885950    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.886371 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.886310    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.886371 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.886320    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.886694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.886597    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.888335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.888276    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.889040 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.889019    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:44.961854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:44.948534    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzcqs\" (UniqueName: \"kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs\") pod \"simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") " pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:45.061493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.061448    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 31 00:43:45.061672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.061514    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 31 00:43:45.078854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.078821    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:45.151770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.151136    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 00:43:45.151770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.151199    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 00:43:45.216822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.215872    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:45.216994 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.216832    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:45.489181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.488137    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"]
Oct 31 00:43:45.490159 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:45.490129    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5755e21_767b_4780_8a8a_093aeaa892ba.slice/crio-f9eacf33ebd44272e5df1f269a886531c6bf4523bd82a2839b1844d3a6679f23 WatchSource:0}: Error finding container f9eacf33ebd44272e5df1f269a886531c6bf4523bd82a2839b1844d3a6679f23: Status 404 returned error can't find the container with id f9eacf33ebd44272e5df1f269a886531c6bf4523bd82a2839b1844d3a6679f23
Oct 31 00:43:45.667175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.667146    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:46792->10.132.0.57:15021: read: connection reset by peer" start-of-body=
Oct 31 00:43:45.667364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.667207    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:46792->10.132.0.57:15021: read: connection reset by peer"
Oct 31 00:43:45.687674 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.687651    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:43:45.784412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.784242    4924 generic.go:356] "Generic (PLEG): container finished" podID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerID="ce4b1a13c849fcc6eb2c8fe71cf1c7c698c9cfb3e806d5adb81960689a74d0f2" exitCode=0
Oct 31 00:43:45.784412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.784340    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" event={"ID":"a2b570d4-6c8e-4454-a84a-37c4c4118471","Type":"ContainerDied","Data":"ce4b1a13c849fcc6eb2c8fe71cf1c7c698c9cfb3e806d5adb81960689a74d0f2"}
Oct 31 00:43:45.784826 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.784803    4924 scope.go:122] "RemoveContainer" containerID="ce4b1a13c849fcc6eb2c8fe71cf1c7c698c9cfb3e806d5adb81960689a74d0f2"
Oct 31 00:43:45.786950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.786920    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:45.787169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.787145    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" event={"ID":"c5755e21-767b-4780-8a8a-093aeaa892ba","Type":"ContainerStarted","Data":"54e50c60c8fc3d45db270772a102bb7ae30d1898dc93a7d615f8ba7a656f8e3d"}
Oct 31 00:43:45.787272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.787182    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" event={"ID":"c5755e21-767b-4780-8a8a-093aeaa892ba","Type":"ContainerStarted","Data":"f9eacf33ebd44272e5df1f269a886531c6bf4523bd82a2839b1844d3a6679f23"}
Oct 31 00:43:45.787272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.787143    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:45.787272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.787257    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:43:45.871660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:45.871189    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podStartSLOduration=1.871172494 podStartE2EDuration="1.871172494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:45.861281729 +0000 UTC m=+626.232065524" watchObservedRunningTime="2026-10-31 00:43:45.871172494 +0000 UTC m=+626.241956286"
Oct 31 00:43:46.075863 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:46.070603    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 31 00:43:46.075863 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:46.070663    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 31 00:43:46.081670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:46.079697    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:46.164595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:46.164138    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 00:43:46.164595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:46.164196    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 00:43:46.822916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:46.816576    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" event={"ID":"a2b570d4-6c8e-4454-a84a-37c4c4118471","Type":"ContainerStarted","Data":"342028bb9114c6b178acae2c95934f13cd0ad5b1036e5f56dd2c98ea23a6aabf"}
Oct 31 00:43:47.062287 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.062248    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 31 00:43:47.062599 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.062562    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 31 00:43:47.151826 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.151166    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 00:43:47.151826 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.151238    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 00:43:47.779771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.776801    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 31 00:43:47.779771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.776861    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 31 00:43:47.808769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:47.796049    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:48.063766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:48.063052    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 31 00:43:48.063766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:48.063138    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 31 00:43:48.148769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:48.148692    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 00:43:48.148952 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:48.148776    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 00:43:48.774550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:48.774515    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 31 00:43:48.774723 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:48.774574    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 31 00:43:49.065737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:49.065658    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:49.066689 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:49.066670    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:43:49.153934 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:49.153904    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:49.155525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:49.155501    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:43:49.775375 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:49.775338    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 31 00:43:49.775561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:49.775393    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 31 00:43:50.774359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:50.774325    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 31 00:43:50.774788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:50.774378    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 31 00:43:51.775181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:51.775142    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 31 00:43:51.775668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:51.775198    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 31 00:43:52.778426 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:52.778395    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:52.778872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:52.778719    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:52.779542 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:52.779521    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:43:54.666912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.666866    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"]
Oct 31 00:43:54.692903 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.692872    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"]
Oct 31 00:43:54.693264 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.693199    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.710008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.709975    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f99ac0\"/\"simulator-w0-f99ac0-kserve-self-signed-certs\""
Oct 31 00:43:54.710203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.710175    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f99ac0\"/\"default-dockercfg-cpj7p\""
Oct 31 00:43:54.711449 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.711426    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f99ac0\"/\"kube-root-ca.crt\""
Oct 31 00:43:54.711682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.711658    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f99ac0\"/\"openshift-service-ca.crt\""
Oct 31 00:43:54.827304 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.827256    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.827523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.827321    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.827523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.827364    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmxt4\" (UniqueName: \"kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.827712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.827685    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.827790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.827777    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.827874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.827851    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.929333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.929238    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.929333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.929312    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.929566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.929344    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.929566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.929449    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.929566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.929476    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.929566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.929514    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmxt4\" (UniqueName: \"kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.930049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.930009    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.930367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.930345    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.930481 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.930446    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.933351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.933320    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.933595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.933574    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:54.946712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:54.946665    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmxt4\" (UniqueName: \"kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4\") pod \"simulator-w0-f99ac0-kserve-594dc4bcf-wswbk\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") " pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:55.014798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.014731    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:55.079885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.079695    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:55.102826 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.100536    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:55.316364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.316327    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"]
Oct 31 00:43:55.897952 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.897919    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" event={"ID":"b2719124-a43f-451b-b627-e4e20d8d8fd3","Type":"ContainerStarted","Data":"6abc6027a9bc93a7bccbf7cf2ed5411e20eb139c7999c60e07c722926070514c"}
Oct 31 00:43:55.898397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.897963    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" event={"ID":"b2719124-a43f-451b-b627-e4e20d8d8fd3","Type":"ContainerStarted","Data":"542f7e89e02a2b28ef3d5cf5be67f8fa2a4746d1cb17f5a005480f6f115e365e"}
Oct 31 00:43:55.909577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.909544    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:43:55.924129 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:55.924076    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podStartSLOduration=1.924062378 podStartE2EDuration="1.924062378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:55.921149576 +0000 UTC m=+636.291933365" watchObservedRunningTime="2026-10-31 00:43:55.924062378 +0000 UTC m=+636.294846163"
Oct 31 00:43:56.015733 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:56.015702    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:43:57.853923 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:43:57.853885    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:43:58.783082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.783043    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"]
Oct 31 00:43:58.787304 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.787272    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.794305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.794254    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7f1496\"/\"openshift-service-ca.crt\""
Oct 31 00:43:58.794465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.794325    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7f1496\"/\"kube-root-ca.crt\""
Oct 31 00:43:58.801398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.801322    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7f1496\"/\"default-dockercfg-zqzc6\""
Oct 31 00:43:58.803047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.803023    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"]
Oct 31 00:43:58.803554 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.803533    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7f1496\"/\"premium-w1-7f1496-kserve-self-signed-certs\""
Oct 31 00:43:58.875027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.874989    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.875411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.875050    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.875411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.875154    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.875411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.875185    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.875411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.875400    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.875548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.875477    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qtqt\" (UniqueName: \"kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976269 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976233    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976278    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976295    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976384    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976437    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qtqt\" (UniqueName: \"kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976457    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976759    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976853 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976808    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.976894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.976848    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.978564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.978542    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.978904 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.978887    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:58.985445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:58.985421    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qtqt\" (UniqueName: \"kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt\") pod \"premium-w1-7f1496-kserve-6f6b97847b-n8kfj\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") " pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:59.104771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:59.104656    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:43:59.393387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:59.393352    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"]
Oct 31 00:43:59.394586 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:43:59.394553    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17009d78_c2f2_496f_b820_2b4765f7b5f1.slice/crio-1fe8db3dbf5644196fc96d25f6080bcd9e5a64273d566bbdc44f4250d7f5589f WatchSource:0}: Error finding container 1fe8db3dbf5644196fc96d25f6080bcd9e5a64273d566bbdc44f4250d7f5589f: Status 404 returned error can't find the container with id 1fe8db3dbf5644196fc96d25f6080bcd9e5a64273d566bbdc44f4250d7f5589f
Oct 31 00:43:59.928730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:59.928692    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" event={"ID":"17009d78-c2f2-496f-b820-2b4765f7b5f1","Type":"ContainerStarted","Data":"8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a"}
Oct 31 00:43:59.928730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:59.928738    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" event={"ID":"17009d78-c2f2-496f-b820-2b4765f7b5f1","Type":"ContainerStarted","Data":"1fe8db3dbf5644196fc96d25f6080bcd9e5a64273d566bbdc44f4250d7f5589f"}
Oct 31 00:43:59.973162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:43:59.973098    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podStartSLOduration=1.97308306 podStartE2EDuration="1.97308306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:59.970267467 +0000 UTC m=+640.341051256" watchObservedRunningTime="2026-10-31 00:43:59.97308306 +0000 UTC m=+640.343866850"
Oct 31 00:44:00.104819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:00.104782    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:44:05.015398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:05.015364    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:44:05.028124 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:05.028102    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:44:05.977496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:05.977467    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:44:09.105252 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:09.105209    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:44:09.119830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:09.119788    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:44:10.003697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:10.003663    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:44:12.625546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.625501    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"]
Oct 31 00:44:12.629624 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.629601    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.639450 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.639426    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-5c2a03-openshift-default-dockercfg-6267p\""
Oct 31 00:44:12.643139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.643111    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"]
Oct 31 00:44:12.741347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741313    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741425    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741464    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqhkt\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741495    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741741 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741625    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741763    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741804    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.741951 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741921    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.742094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.741973    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843100    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843160    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843197    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843217    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqhkt\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843360    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843437    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843528    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843554    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843599    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843657    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.843819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.843759    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.844050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.844030    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.844137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.844114    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.844407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.844385    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.845673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.845654    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.845831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.845815    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.851805 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.851785    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.852261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.852240    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqhkt\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk\" (UID: \"914a5656-be09-46ed-8413-633981153991\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:12.947068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:12.946691    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:13.241722 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:13.241637    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"]
Oct 31 00:44:13.245219 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:44:13.245186    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914a5656_be09_46ed_8413_633981153991.slice/crio-aad8da2f607c87fee7e4339233f6455e2028c8ed909ed29b7a7887251aff9ccb WatchSource:0}: Error finding container aad8da2f607c87fee7e4339233f6455e2028c8ed909ed29b7a7887251aff9ccb: Status 404 returned error can't find the container with id aad8da2f607c87fee7e4339233f6455e2028c8ed909ed29b7a7887251aff9ccb
Oct 31 00:44:13.247709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:13.247670    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:13.247838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:13.247738    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:13.247838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:13.247783    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:14.035197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.035159    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" event={"ID":"914a5656-be09-46ed-8413-633981153991","Type":"ContainerStarted","Data":"50dc87b7e3d795eda116139637cb51b50184fab1597b6d9eb45b98e67ba41f3f"}
Oct 31 00:44:14.035716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.035219    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" event={"ID":"914a5656-be09-46ed-8413-633981153991","Type":"ContainerStarted","Data":"aad8da2f607c87fee7e4339233f6455e2028c8ed909ed29b7a7887251aff9ccb"}
Oct 31 00:44:14.056626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.056561    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podStartSLOduration=2.056540108 podStartE2EDuration="2.056540108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:14.054592054 +0000 UTC m=+654.425375844" watchObservedRunningTime="2026-10-31 00:44:14.056540108 +0000 UTC m=+654.427323899"
Oct 31 00:44:14.947607 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.947569    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:14.948943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.948914    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:44:14.949092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.948970    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 00:44:14.971774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.971730    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"]
Oct 31 00:44:14.976143 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.976116    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:14.988662 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.988637    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d4a7e\"/\"premium-w3-1d4a7e-kserve-self-signed-certs\""
Oct 31 00:44:14.989653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:14.989626    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"]
Oct 31 00:44:15.068605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.068572    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.069043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.068623    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.069043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.068646    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.069043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.068885    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.069043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.068951    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpvxr\" (UniqueName: \"kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.069043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.069002    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.170475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.170432    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.170680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.170499    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.170680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.170630    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.170953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.170924    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.171044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.171013    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpvxr\" (UniqueName: \"kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.171106 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.171069    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.171509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.170927    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.171509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.171430    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.171654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.171564    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.173772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.173735    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.174046 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.174027    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.188155 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.188124    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpvxr\" (UniqueName: \"kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr\") pod \"premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") " pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.289354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.289318    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:15.591215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.591168    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"]
Oct 31 00:44:15.947962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.947873    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:44:15.948110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:15.947969    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 00:44:16.050546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:16.050502    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" event={"ID":"472a32a4-989d-4dc1-a65a-4a8267ef9283","Type":"ContainerStarted","Data":"bb29103e6a8f0307228b7b7bf4e459917216b7a40b7d089bdc59258538278290"}
Oct 31 00:44:16.050797 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:16.050559    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" event={"ID":"472a32a4-989d-4dc1-a65a-4a8267ef9283","Type":"ContainerStarted","Data":"78b6243b0c914be05dc5510c2ed1801d16542cf797c42a867663b080633b08dc"}
Oct 31 00:44:16.072081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:16.072014    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podStartSLOduration=2.071994904 podStartE2EDuration="2.071994904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:16.067610657 +0000 UTC m=+656.438394445" watchObservedRunningTime="2026-10-31 00:44:16.071994904 +0000 UTC m=+656.442778694"
Oct 31 00:44:16.290122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:16.290088    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:16.947898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:16.947859    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:44:16.948101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:16.947918    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 00:44:17.948069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:17.948032    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:44:17.948518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:17.948092    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 00:44:18.948011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:18.947976    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:44:18.948204 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:18.948033    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 00:44:20.168781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:20.168735    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:20.169366 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:20.169122    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:20.169965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:20.169943    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:44:23.670989 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.670951    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"]
Oct 31 00:44:23.677072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.677046    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.700545 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.700516    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f99ac0\"/\"premium-w0-f99ac0-kserve-self-signed-certs\""
Oct 31 00:44:23.702690 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.702662    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"]
Oct 31 00:44:23.800332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.800049    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.800332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.800113    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trsmj\" (UniqueName: \"kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.800332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.800194    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.800332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.800264    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.800683 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.800383    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.800683 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.800474    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.902145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902080    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.902353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902184    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.902353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902323    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.902480 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902351    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trsmj\" (UniqueName: \"kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.902480 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902401    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.902480 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902454    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.903188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902822    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.903188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902824    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.903188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.902987    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.904886 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.904858    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.905679 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.905636    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.915132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.915100    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trsmj\" (UniqueName: \"kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj\") pod \"premium-w0-f99ac0-kserve-7db85fb86-wtr5b\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") " pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:23.992284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:23.992177    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:24.304708 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:24.304667    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"]
Oct 31 00:44:24.310887 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:44:24.310859    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafcac907_e8a5_4b38_9523_01f69b0a2058.slice/crio-80ffea2bfad3da51c0e3f2446e76ae699b7a16406ee9b3893d989fd5b1929110 WatchSource:0}: Error finding container 80ffea2bfad3da51c0e3f2446e76ae699b7a16406ee9b3893d989fd5b1929110: Status 404 returned error can't find the container with id 80ffea2bfad3da51c0e3f2446e76ae699b7a16406ee9b3893d989fd5b1929110
Oct 31 00:44:25.120632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:25.120595    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" event={"ID":"afcac907-e8a5-4b38-9523-01f69b0a2058","Type":"ContainerStarted","Data":"33eec5c6f708d8d486d6733b44b2dca320f7cdecde4b46a86fcfa20fe063d6ff"}
Oct 31 00:44:25.120632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:25.120643    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" event={"ID":"afcac907-e8a5-4b38-9523-01f69b0a2058","Type":"ContainerStarted","Data":"80ffea2bfad3da51c0e3f2446e76ae699b7a16406ee9b3893d989fd5b1929110"}
Oct 31 00:44:25.139838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:25.139787    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podStartSLOduration=2.139773852 podStartE2EDuration="2.139773852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:25.136671373 +0000 UTC m=+665.507455164" watchObservedRunningTime="2026-10-31 00:44:25.139773852 +0000 UTC m=+665.510557640"
Oct 31 00:44:25.289821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:25.289783    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:25.308117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:25.308085    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:25.992541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:25.992507    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:26.139093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:26.139063    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:44:27.854120 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:44:27.854086    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:44:34.011827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:34.005338    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:34.052695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:34.052658    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:34.208975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:34.208937    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:44:42.488033 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.487996    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"]
Oct 31 00:44:42.492072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.492053    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.503595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.503575    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-44bc67-openshift-default-dockercfg-z6hg4\""
Oct 31 00:44:42.526986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.526952    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"]
Oct 31 00:44:42.588923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.588885    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9glgr\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.588923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.588927    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.588949    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.589033    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.589186    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.589217    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589346 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.589242    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589346 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.589320    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.589424 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.589375    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690327 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690293    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690334    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690357    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690381    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690519    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690600    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9glgr\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690637    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690665    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690701    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.690888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.690847    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.691273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.691244    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.691412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.691293    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.691412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.691308    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.691545 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.691448    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.693730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.693694    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.694619 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.694595    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.701217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.701191    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.701301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.701198    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9glgr\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:42.810035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:42.809997    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:43.097731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.097637    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"]
Oct 31 00:44:43.101139 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:44:43.101107    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0ad9de_782c_41ae_87af_ea375f20c260.slice/crio-af4bee1744a7648c76ecb9e582df701542a6b7b39c76cb33f277f8db2dcb97b3 WatchSource:0}: Error finding container af4bee1744a7648c76ecb9e582df701542a6b7b39c76cb33f277f8db2dcb97b3: Status 404 returned error can't find the container with id af4bee1744a7648c76ecb9e582df701542a6b7b39c76cb33f277f8db2dcb97b3
Oct 31 00:44:43.104140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.104101    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:43.104229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.104176    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:43.104229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.104206    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:43.268632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.268595    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" event={"ID":"6f0ad9de-782c-41ae-87af-ea375f20c260","Type":"ContainerStarted","Data":"35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940"}
Oct 31 00:44:43.268849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.268656    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" event={"ID":"6f0ad9de-782c-41ae-87af-ea375f20c260","Type":"ContainerStarted","Data":"af4bee1744a7648c76ecb9e582df701542a6b7b39c76cb33f277f8db2dcb97b3"}
Oct 31 00:44:43.304948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.304873    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podStartSLOduration=1.30485378 podStartE2EDuration="1.30485378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:43.299487235 +0000 UTC m=+683.670271025" watchObservedRunningTime="2026-10-31 00:44:43.30485378 +0000 UTC m=+683.675637570"
Oct 31 00:44:43.822936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:43.822892    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:44.830142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:44.827917    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 00:44:44.830142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:44.828023    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 00:44:45.812188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:45.812146    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 00:44:45.812475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:45.812442    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 00:44:46.814770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:46.814167    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 00:44:46.814770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:46.814230    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 00:44:47.826785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:47.826532    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 00:44:47.826785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:47.826606    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 00:44:48.812777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:48.812183    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 00:44:48.812777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:48.812285    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 00:44:49.811301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:49.811258    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 00:44:49.811711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:49.811367    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 00:44:51.027144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:51.027115    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:51.028169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:51.028001    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:51.029033 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:51.029011    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:44:57.832821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.832787    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"]
Oct 31 00:44:57.838821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.838792    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.854474 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.854382    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"]
Oct 31 00:44:57.855432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.855273    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-7f1496-openshift-default-dockercfg-jpdcs\""
Oct 31 00:44:57.857988 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:44:57.857953    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:44:57.971866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.971833    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.971873    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.971989    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.972094    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.972149    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.972176    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.972254    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqg8x\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.972319    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:57.972435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:57.972359    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073303 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073269    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073311    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073497    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqg8x\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073532    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073564    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073668    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073705    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073817    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073875    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.073996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.073969    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.074150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.074123    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.074280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.074174    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.074280 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.074262    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.074406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.074360    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.075684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.075660    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.076033 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.076015    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.081310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.081287    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.081406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.081316    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqg8x\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.156233 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.156133    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:44:58.446753 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.446712    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"]
Oct 31 00:44:58.447505 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:44:58.447481    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc13c58cb_6288_4c94_82b9_8ef44e90c10d.slice/crio-7b915bda35d011a7a908ee7c95cab512c973e15be4ec7285ad852a5fd276adb1 WatchSource:0}: Error finding container 7b915bda35d011a7a908ee7c95cab512c973e15be4ec7285ad852a5fd276adb1: Status 404 returned error can't find the container with id 7b915bda35d011a7a908ee7c95cab512c973e15be4ec7285ad852a5fd276adb1
Oct 31 00:44:58.449966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.449933    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:58.450045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.449998    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:58.450045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:58.450029    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:44:59.444974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:59.444932    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" event={"ID":"c13c58cb-6288-4c94-82b9-8ef44e90c10d","Type":"ContainerStarted","Data":"fe7a23eef0a729873c1c34ef15a348b8c0440f8359827c5df857081028a71ba5"}
Oct 31 00:44:59.445444 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:59.445005    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" event={"ID":"c13c58cb-6288-4c94-82b9-8ef44e90c10d","Type":"ContainerStarted","Data":"7b915bda35d011a7a908ee7c95cab512c973e15be4ec7285ad852a5fd276adb1"}
Oct 31 00:44:59.467514 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:44:59.467459    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podStartSLOduration=2.467446213 podStartE2EDuration="2.467446213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:59.465136951 +0000 UTC m=+699.835920742" watchObservedRunningTime="2026-10-31 00:44:59.467446213 +0000 UTC m=+699.838230002"
Oct 31 00:45:00.156347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.156315    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:45:00.157863 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.157832    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:45:00.158009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.157894    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 00:45:00.166506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.166475    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"]
Oct 31 00:45:00.170902 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.170873    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.171686 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.171664    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"]
Oct 31 00:45:00.178794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.178736    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"]
Oct 31 00:45:00.185104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.185078    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:45:00.185862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.185844    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.222991 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.222956    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"]
Oct 31 00:45:00.223266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.223251    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"]
Oct 31 00:45:00.223383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.223372    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"]
Oct 31 00:45:00.223563 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.223551    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.248798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.248595    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"]
Oct 31 00:45:00.258694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.255051    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.267333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.266427    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"]
Oct 31 00:45:00.275337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.274994    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.283133 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.280827    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"]
Oct 31 00:45:00.283133 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.282176    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"]
Oct 31 00:45:00.335788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.334710    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.335788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.334804    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92rz9\" (UniqueName: \"kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9\") pod \"maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.335788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.334885    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5xf4\" (UniqueName: \"kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4\") pod \"maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.335788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.334960    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n97fc\" (UniqueName: \"kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc\") pod \"maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.335788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.335002    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.335788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.335041    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.437166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.436969    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.437166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437148    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x96sj\" (UniqueName: \"kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj\") pod \"maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.437393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437196    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.437393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437261    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmb9p\" (UniqueName: \"kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p\") pod \"maas-api-key-cleanup-29890125-h55sd\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.437393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437330    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.437393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437359    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890125-h55sd\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.437597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437407    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92rz9\" (UniqueName: \"kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9\") pod \"maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.437597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437435    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.437597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437524    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5xf4\" (UniqueName: \"kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4\") pod \"maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.437801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.437610    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n97fc\" (UniqueName: \"kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc\") pod \"maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.438688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.438663    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.438874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.438712    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.439172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.439130    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.450140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.450108    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5xf4\" (UniqueName: \"kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4\") pod \"maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.456303 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.456271    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n97fc\" (UniqueName: \"kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc\") pod \"maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.456886 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.456802    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92rz9\" (UniqueName: \"kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9\") pod \"maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.538511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.538462    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x96sj\" (UniqueName: \"kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj\") pod \"maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.538688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.538547    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmb9p\" (UniqueName: \"kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p\") pod \"maas-api-key-cleanup-29890125-h55sd\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.538688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.538615    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890125-h55sd\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.538688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.538654    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.539573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.539535    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.540070 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.540031    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890125-h55sd\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.559685 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.559654    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:00.572912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.572873    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x96sj\" (UniqueName: \"kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj\") pod \"maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.573945 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.573917    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmb9p\" (UniqueName: \"kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p\") pod \"maas-api-key-cleanup-29890125-h55sd\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.582118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.581936    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:00.646124 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.645913    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:00.652142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.652118    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:00.659257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:00.658924    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:00.990913 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:00.990868    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2867d9_b020_4a26_b76c_6fb89d073901.slice/crio-54078018532ed162f691627426a58ae472be1713b836ff728b3c61bf7cf91cf1 WatchSource:0}: Error finding container 54078018532ed162f691627426a58ae472be1713b836ff728b3c61bf7cf91cf1: Status 404 returned error can't find the container with id 54078018532ed162f691627426a58ae472be1713b836ff728b3c61bf7cf91cf1
Oct 31 00:45:01.029811 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.015318    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"]
Oct 31 00:45:01.140077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.139136    4924 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:45:01.156550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.156521    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:45:01.156741 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.156716    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 00:45:01.352871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.352824    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"]
Oct 31 00:45:01.365437 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:01.358690    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod359b6fb8_0144_4830_9171_21e5a3d888cb.slice/crio-daa2adbcf6ee20150cd145c7e40ed132409875b8106cc45c2b6755b9f3703db2 WatchSource:0}: Error finding container daa2adbcf6ee20150cd145c7e40ed132409875b8106cc45c2b6755b9f3703db2: Status 404 returned error can't find the container with id daa2adbcf6ee20150cd145c7e40ed132409875b8106cc45c2b6755b9f3703db2
Oct 31 00:45:01.470152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.455137    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"]
Oct 31 00:45:01.496778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.489276    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"]
Oct 31 00:45:01.509309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.509270    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerStarted","Data":"daa2adbcf6ee20150cd145c7e40ed132409875b8106cc45c2b6755b9f3703db2"}
Oct 31 00:45:01.555567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.555527    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerStarted","Data":"a82496fa07d5e9a96ea1874404a42443f4a9e24dd5350a370af7583629dc322f"}
Oct 31 00:45:01.556903 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.556877    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"]
Oct 31 00:45:01.566612 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:01.566457    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerStarted","Data":"54078018532ed162f691627426a58ae472be1713b836ff728b3c61bf7cf91cf1"}
Oct 31 00:45:02.158231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:02.158196    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:45:02.158413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:02.158252    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 00:45:02.577689 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:02.577630    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerStarted","Data":"58b4c85755f02ee9336cca9d93af89ffc26b0d8ad3bac37bb4de5f1d3fca71e5"}
Oct 31 00:45:02.581417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:02.581376    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" event={"ID":"f68da95a-8b68-4c68-a555-308b36413d92","Type":"ContainerStarted","Data":"1373c1049ce7bcdf06696d11ae0d66bac8d366819be41e7b57b42e06a606aaa4"}
Oct 31 00:45:03.163699 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:03.157239    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:45:03.163699 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:03.157302    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 00:45:04.157721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:04.157681    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:45:04.158236 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:04.157777    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 00:45:05.158452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.158410    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:45:05.159045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.158487    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 00:45:05.317604 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.317547    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"]
Oct 31 00:45:05.337830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.336517    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.345520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.345481    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:45:05.347294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.346664    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"]
Oct 31 00:45:05.451553 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.451461    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfslv\" (UniqueName: \"kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.451553 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.451539    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.451735 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.451684    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.553602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.552872    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfslv\" (UniqueName: \"kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.553602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.552935    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.553602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.553088    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.555204 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.554911    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.578917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.578850    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.611195 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.600238    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfslv\" (UniqueName: \"kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv\") pod \"maas-api-revoke-sub-6205a64c-n2g5w\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:05.668323 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:05.668115    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:06.091788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:06.091713    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"]
Oct 31 00:45:06.092446 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:06.092404    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce8f095_7fc6_4094_885d_874962f41717.slice/crio-fbdcc08c5397dafb04191d83ad0dc21ba77d423917b9c596c5a8adef186782f5 WatchSource:0}: Error finding container fbdcc08c5397dafb04191d83ad0dc21ba77d423917b9c596c5a8adef186782f5: Status 404 returned error can't find the container with id fbdcc08c5397dafb04191d83ad0dc21ba77d423917b9c596c5a8adef186782f5
Oct 31 00:45:06.162837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:06.162801    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:45:06.164164 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:06.164137    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:45:06.168359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:06.167871    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:45:06.687322 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:06.687278    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w" event={"ID":"bce8f095-7fc6-4094-885d-874962f41717","Type":"ContainerStarted","Data":"fbdcc08c5397dafb04191d83ad0dc21ba77d423917b9c596c5a8adef186782f5"}
Oct 31 00:45:08.714272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.714232    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerStarted","Data":"5fad7c483fc90a954987e989057dd21bb28db5b2aefd253090509d2101bfbfe4"}
Oct 31 00:45:08.716552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.716094    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerStarted","Data":"695b2387c486edd0228588d173970fd0430e2bafa70fc0f908b527ec50a41f29"}
Oct 31 00:45:08.717963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.717937    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerStarted","Data":"77ca73ad87ec23b339dfe0139e37d83b8ac0ede878d78270de680417cf8a5083"}
Oct 31 00:45:08.719458 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.719437    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerStarted","Data":"78a2aff6981f835fa02acaeda43b39fdba079be3529b303aa3d9c66e1c799e08"}
Oct 31 00:45:08.733454 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.733397    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" podStartSLOduration=2.422445335 podStartE2EDuration="8.733378364s" totalImagesPullingTime="6.310933029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:08.728582033 +0000 UTC m=+709.099365823" watchObservedRunningTime="2026-10-31 00:45:08.733378364 +0000 UTC m=+709.104162154"
Oct 31 00:45:08.745100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.745057    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" podStartSLOduration=1.9871271849999999 podStartE2EDuration="8.745043918s" totalImagesPullingTime="6.757916733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:08.743710267 +0000 UTC m=+709.114494057" watchObservedRunningTime="2026-10-31 00:45:08.745043918 +0000 UTC m=+709.115827707"
Oct 31 00:45:08.762608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.762511    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" podStartSLOduration=2.335910152 podStartE2EDuration="8.762498622s" totalImagesPullingTime="6.42658847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:08.760146852 +0000 UTC m=+709.130930654" watchObservedRunningTime="2026-10-31 00:45:08.762498622 +0000 UTC m=+709.133282410"
Oct 31 00:45:08.776729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:08.776679    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" podStartSLOduration=2.201302799 podStartE2EDuration="8.776666591s" totalImagesPullingTime="6.575363792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:08.77565947 +0000 UTC m=+709.146443258" watchObservedRunningTime="2026-10-31 00:45:08.776666591 +0000 UTC m=+709.147450380"
Oct 31 00:45:09.730518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.730489    4924 generic.go:356] "Generic (PLEG): container finished" podID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerID="5fad7c483fc90a954987e989057dd21bb28db5b2aefd253090509d2101bfbfe4" exitCode=60
Oct 31 00:45:09.730973 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.730579    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerDied","Data":"5fad7c483fc90a954987e989057dd21bb28db5b2aefd253090509d2101bfbfe4"}
Oct 31 00:45:09.730973 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.730869    4924 scope.go:122] "RemoveContainer" containerID="5fad7c483fc90a954987e989057dd21bb28db5b2aefd253090509d2101bfbfe4"
Oct 31 00:45:09.732690 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.732663    4924 generic.go:356] "Generic (PLEG): container finished" podID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerID="695b2387c486edd0228588d173970fd0430e2bafa70fc0f908b527ec50a41f29" exitCode=60
Oct 31 00:45:09.732824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.732775    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerDied","Data":"695b2387c486edd0228588d173970fd0430e2bafa70fc0f908b527ec50a41f29"}
Oct 31 00:45:09.733089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.733068    4924 scope.go:122] "RemoveContainer" containerID="695b2387c486edd0228588d173970fd0430e2bafa70fc0f908b527ec50a41f29"
Oct 31 00:45:09.736729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.736704    4924 generic.go:356] "Generic (PLEG): container finished" podID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerID="77ca73ad87ec23b339dfe0139e37d83b8ac0ede878d78270de680417cf8a5083" exitCode=60
Oct 31 00:45:09.736839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.736799    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerDied","Data":"77ca73ad87ec23b339dfe0139e37d83b8ac0ede878d78270de680417cf8a5083"}
Oct 31 00:45:09.737078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.737061    4924 scope.go:122] "RemoveContainer" containerID="77ca73ad87ec23b339dfe0139e37d83b8ac0ede878d78270de680417cf8a5083"
Oct 31 00:45:09.739059 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.739038    4924 generic.go:356] "Generic (PLEG): container finished" podID="bce8f095-7fc6-4094-885d-874962f41717" containerID="a8dbf56993e4b6aab83f8523969aea2fe1683feb0ddda14edc358813a9b82b4b" exitCode=0
Oct 31 00:45:09.739186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.739162    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w" event={"ID":"bce8f095-7fc6-4094-885d-874962f41717","Type":"ContainerDied","Data":"a8dbf56993e4b6aab83f8523969aea2fe1683feb0ddda14edc358813a9b82b4b"}
Oct 31 00:45:09.741121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.741100    4924 generic.go:356] "Generic (PLEG): container finished" podID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerID="78a2aff6981f835fa02acaeda43b39fdba079be3529b303aa3d9c66e1c799e08" exitCode=60
Oct 31 00:45:09.741217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.741182    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerDied","Data":"78a2aff6981f835fa02acaeda43b39fdba079be3529b303aa3d9c66e1c799e08"}
Oct 31 00:45:09.741411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.741393    4924 scope.go:122] "RemoveContainer" containerID="78a2aff6981f835fa02acaeda43b39fdba079be3529b303aa3d9c66e1c799e08"
Oct 31 00:45:09.742905 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.742883    4924 generic.go:356] "Generic (PLEG): container finished" podID="f68da95a-8b68-4c68-a555-308b36413d92" containerID="daae73e42912aace71bfa5652a4344e2bcb91e0d2d01059ef308846568a9a298" exitCode=60
Oct 31 00:45:09.742993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.742958    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" event={"ID":"f68da95a-8b68-4c68-a555-308b36413d92","Type":"ContainerDied","Data":"daae73e42912aace71bfa5652a4344e2bcb91e0d2d01059ef308846568a9a298"}
Oct 31 00:45:09.743179 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:09.743157    4924 scope.go:122] "RemoveContainer" containerID="daae73e42912aace71bfa5652a4344e2bcb91e0d2d01059ef308846568a9a298"
Oct 31 00:45:10.753198 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:10.753102    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerStarted","Data":"0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4"}
Oct 31 00:45:10.759732 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:10.759699    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerStarted","Data":"c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9"}
Oct 31 00:45:10.763874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:10.763834    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerStarted","Data":"640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff"}
Oct 31 00:45:10.778626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:10.778585    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" event={"ID":"f68da95a-8b68-4c68-a555-308b36413d92","Type":"ContainerStarted","Data":"7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd"}
Oct 31 00:45:10.783522 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:10.783489    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerStarted","Data":"5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442"}
Oct 31 00:45:10.978488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:10.967567    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" podStartSLOduration=4.415695799 podStartE2EDuration="10.967542899s" totalImagesPullingTime="6.5518471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:10.895440724 +0000 UTC m=+711.266224514" watchObservedRunningTime="2026-10-31 00:45:10.967542899 +0000 UTC m=+711.338326689"
Oct 31 00:45:11.254465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.251164    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:11.422822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.415576    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca\") pod \"bce8f095-7fc6-4094-885d-874962f41717\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") "
Oct 31 00:45:11.422822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.415674    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv\" (UniqueName: \"kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv\") pod \"bce8f095-7fc6-4094-885d-874962f41717\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") "
Oct 31 00:45:11.422822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.415883    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca\") pod \"bce8f095-7fc6-4094-885d-874962f41717\" (UID: \"bce8f095-7fc6-4094-885d-874962f41717\") "
Oct 31 00:45:11.437772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.428375    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca" pod "bce8f095-7fc6-4094-885d-874962f41717" (UID: "bce8f095-7fc6-4094-885d-874962f41717"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:11.446721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.441282    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca" pod "bce8f095-7fc6-4094-885d-874962f41717" (UID: "bce8f095-7fc6-4094-885d-874962f41717"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:11.446721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.446082    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv" pod "bce8f095-7fc6-4094-885d-874962f41717" (UID: "bce8f095-7fc6-4094-885d-874962f41717"). InnerVolumeSpecName "kube-api-access-pfslv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:11.521771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.520805    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bce8f095-7fc6-4094-885d-874962f41717-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:11.521771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.520860    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bce8f095-7fc6-4094-885d-874962f41717-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:11.521771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.520877    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfslv\" (UniqueName: \"kubernetes.io/projected/bce8f095-7fc6-4094-885d-874962f41717-kube-api-access-pfslv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:11.847768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.846280    4924 generic.go:356] "Generic (PLEG): container finished" podID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerID="5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442" exitCode=60
Oct 31 00:45:11.847768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.846448    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerDied","Data":"5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442"}
Oct 31 00:45:11.847768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.846510    4924 scope.go:122] "RemoveContainer" containerID="5fad7c483fc90a954987e989057dd21bb28db5b2aefd253090509d2101bfbfe4"
Oct 31 00:45:11.853103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.851013    4924 scope.go:122] "RemoveContainer" containerID="5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442"
Oct 31 00:45:11.853103 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:11.851344    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6_odh-ai-gateway-infra(359b6fb8-0144-4830-9171-21e5a3d888cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb"
Oct 31 00:45:11.859935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.857540    4924 generic.go:356] "Generic (PLEG): container finished" podID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerID="0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4" exitCode=60
Oct 31 00:45:11.859935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.857692    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerDied","Data":"0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4"}
Oct 31 00:45:11.862635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.860581    4924 scope.go:122] "RemoveContainer" containerID="0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4"
Oct 31 00:45:11.862635 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:11.861605    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6_odh-ai-gateway-infra(d7c8d87f-5619-4988-a7d2-ea084e5b3970)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970"
Oct 31 00:45:11.862635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.862058    4924 generic.go:356] "Generic (PLEG): container finished" podID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerID="c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9" exitCode=60
Oct 31 00:45:11.862635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.862141    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerDied","Data":"c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9"}
Oct 31 00:45:11.862635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.862479    4924 scope.go:122] "RemoveContainer" containerID="c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9"
Oct 31 00:45:11.862971 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:11.862723    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8_odh-ai-gateway-infra(9c2867d9-b020-4a26-b76c-6fb89d073901)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901"
Oct 31 00:45:11.866769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.865291    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w" event={"ID":"bce8f095-7fc6-4094-885d-874962f41717","Type":"ContainerDied","Data":"fbdcc08c5397dafb04191d83ad0dc21ba77d423917b9c596c5a8adef186782f5"}
Oct 31 00:45:11.866769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.865326    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbdcc08c5397dafb04191d83ad0dc21ba77d423917b9c596c5a8adef186782f5"
Oct 31 00:45:11.866769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.865382    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"
Oct 31 00:45:11.870762 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.867626    4924 generic.go:356] "Generic (PLEG): container finished" podID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerID="640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff" exitCode=60
Oct 31 00:45:11.870762 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.867680    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerDied","Data":"640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff"}
Oct 31 00:45:11.870762 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.867988    4924 scope.go:122] "RemoveContainer" containerID="640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff"
Oct 31 00:45:11.870762 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:11.868175    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890125-h55sd_odh-ai-gateway-infra(5f6a71e9-a9e4-4071-b335-4bf48f2e3a23)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23"
Oct 31 00:45:11.887770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.872334    4924 generic.go:356] "Generic (PLEG): container finished" podID="f68da95a-8b68-4c68-a555-308b36413d92" containerID="7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd" exitCode=60
Oct 31 00:45:11.887770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.872450    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" event={"ID":"f68da95a-8b68-4c68-a555-308b36413d92","Type":"ContainerDied","Data":"7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd"}
Oct 31 00:45:11.887770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.872795    4924 scope.go:122] "RemoveContainer" containerID="7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd"
Oct 31 00:45:11.887770 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:11.873033    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj_odh-ai-gateway-infra(f68da95a-8b68-4c68-a555-308b36413d92)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" podUID="f68da95a-8b68-4c68-a555-308b36413d92"
Oct 31 00:45:11.887770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.874976    4924 scope.go:122] "RemoveContainer" containerID="695b2387c486edd0228588d173970fd0430e2bafa70fc0f908b527ec50a41f29"
Oct 31 00:45:11.911879 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.910135    4924 scope.go:122] "RemoveContainer" containerID="77ca73ad87ec23b339dfe0139e37d83b8ac0ede878d78270de680417cf8a5083"
Oct 31 00:45:11.945864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.942944    4924 scope.go:122] "RemoveContainer" containerID="78a2aff6981f835fa02acaeda43b39fdba079be3529b303aa3d9c66e1c799e08"
Oct 31 00:45:11.985672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:11.985627    4924 scope.go:122] "RemoveContainer" containerID="daae73e42912aace71bfa5652a4344e2bcb91e0d2d01059ef308846568a9a298"
Oct 31 00:45:12.885559 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:12.885525    4924 scope.go:122] "RemoveContainer" containerID="5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442"
Oct 31 00:45:12.886069 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:12.885865    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6_odh-ai-gateway-infra(359b6fb8-0144-4830-9171-21e5a3d888cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb"
Oct 31 00:45:12.887978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:12.887945    4924 scope.go:122] "RemoveContainer" containerID="0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4"
Oct 31 00:45:12.888299 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:12.888269    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6_odh-ai-gateway-infra(d7c8d87f-5619-4988-a7d2-ea084e5b3970)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970"
Oct 31 00:45:12.892077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:12.892049    4924 scope.go:122] "RemoveContainer" containerID="c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9"
Oct 31 00:45:12.892482 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:12.892446    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8_odh-ai-gateway-infra(9c2867d9-b020-4a26-b76c-6fb89d073901)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901"
Oct 31 00:45:12.895114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:12.895093    4924 scope.go:122] "RemoveContainer" containerID="640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff"
Oct 31 00:45:12.895440 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:12.895417    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890125-h55sd_odh-ai-gateway-infra(5f6a71e9-a9e4-4071-b335-4bf48f2e3a23)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23"
Oct 31 00:45:12.900769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:12.897220    4924 scope.go:122] "RemoveContainer" containerID="7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd"
Oct 31 00:45:12.900769 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:12.897522    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj_odh-ai-gateway-infra(f68da95a-8b68-4c68-a555-308b36413d92)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" podUID="f68da95a-8b68-4c68-a555-308b36413d92"
Oct 31 00:45:16.142102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.142066    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"]
Oct 31 00:45:16.142677 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.142663    4924 cpu_manager.go:414] "containerMap: removing container" podUID="bce8f095-7fc6-4094-885d-874962f41717" containerName="revoke-keys"
Oct 31 00:45:16.142716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.142678    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bce8f095-7fc6-4094-885d-874962f41717" containerName="revoke-keys"
Oct 31 00:45:16.142808 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.142798    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="bce8f095-7fc6-4094-885d-874962f41717" containerName="revoke-keys"
Oct 31 00:45:16.147484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.147463    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.151498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.151472    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:45:16.155455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.155427    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"]
Oct 31 00:45:16.224213 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.224165    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.224405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.224313    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.224476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.224426    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2bhr\" (UniqueName: \"kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.326032 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.325997    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.326211 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.326097    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2bhr\" (UniqueName: \"kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.326211 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.326172    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.326921 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.326895    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.328734 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.328703    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.334137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.334105    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2bhr\" (UniqueName: \"kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr\") pod \"maas-api-revoke-sub-3fab882f-lg2tl\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.464513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.464426    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:16.769428 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.769396    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"]
Oct 31 00:45:16.771074 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:16.771032    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139c87ed_b8ff_484c_b471_5684a5023a5b.slice/crio-7687b2fae147da716b6845169a510f64e2ab6959b5309b3f8c43da1159ac9d87 WatchSource:0}: Error finding container 7687b2fae147da716b6845169a510f64e2ab6959b5309b3f8c43da1159ac9d87: Status 404 returned error can't find the container with id 7687b2fae147da716b6845169a510f64e2ab6959b5309b3f8c43da1159ac9d87
Oct 31 00:45:16.962937 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:16.962354    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl" event={"ID":"139c87ed-b8ff-484c-b471-5684a5023a5b","Type":"ContainerStarted","Data":"7687b2fae147da716b6845169a510f64e2ab6959b5309b3f8c43da1159ac9d87"}
Oct 31 00:45:17.973561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:17.973526    4924 generic.go:356] "Generic (PLEG): container finished" podID="139c87ed-b8ff-484c-b471-5684a5023a5b" containerID="cabe72de564f5384692d3e427975e16062f7ed7aeb2fde0765e36bf45fcd7360" exitCode=0
Oct 31 00:45:17.974080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:17.973609    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl" event={"ID":"139c87ed-b8ff-484c-b471-5684a5023a5b","Type":"ContainerDied","Data":"cabe72de564f5384692d3e427975e16062f7ed7aeb2fde0765e36bf45fcd7360"}
Oct 31 00:45:19.159963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.159938    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:19.275487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.275451    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr\" (UniqueName: \"kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr\") pod \"139c87ed-b8ff-484c-b471-5684a5023a5b\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") "
Oct 31 00:45:19.275702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.275681    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca\") pod \"139c87ed-b8ff-484c-b471-5684a5023a5b\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") "
Oct 31 00:45:19.275793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.275758    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca\") pod \"139c87ed-b8ff-484c-b471-5684a5023a5b\" (UID: \"139c87ed-b8ff-484c-b471-5684a5023a5b\") "
Oct 31 00:45:19.276144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.276116    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca" pod "139c87ed-b8ff-484c-b471-5684a5023a5b" (UID: "139c87ed-b8ff-484c-b471-5684a5023a5b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:19.276287 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.276274    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:19.278121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.278091    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca" pod "139c87ed-b8ff-484c-b471-5684a5023a5b" (UID: "139c87ed-b8ff-484c-b471-5684a5023a5b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:19.278243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.278136    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr" pod "139c87ed-b8ff-484c-b471-5684a5023a5b" (UID: "139c87ed-b8ff-484c-b471-5684a5023a5b"). InnerVolumeSpecName "kube-api-access-q2bhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:19.296131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.296105    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"]
Oct 31 00:45:19.296664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.296650    4924 cpu_manager.go:414] "containerMap: removing container" podUID="139c87ed-b8ff-484c-b471-5684a5023a5b" containerName="revoke-keys"
Oct 31 00:45:19.296731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.296665    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="139c87ed-b8ff-484c-b471-5684a5023a5b" containerName="revoke-keys"
Oct 31 00:45:19.296809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.296799    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="139c87ed-b8ff-484c-b471-5684a5023a5b" containerName="revoke-keys"
Oct 31 00:45:19.300288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.300269    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.309993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.309969    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"]
Oct 31 00:45:19.377849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.377813    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.378066 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.378043    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.378120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.378081    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfxcg\" (UniqueName: \"kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.378155 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.378147    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/139c87ed-b8ff-484c-b471-5684a5023a5b-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:19.378192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.378157    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2bhr\" (UniqueName: \"kubernetes.io/projected/139c87ed-b8ff-484c-b471-5684a5023a5b-kube-api-access-q2bhr\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:19.479019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.478985    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.479188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.479024    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfxcg\" (UniqueName: \"kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.479188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.479053    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.479635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.479614    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.481329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.481305    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.487922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.487901    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfxcg\" (UniqueName: \"kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg\") pod \"maas-api-revoke-sub-709031f8-tr98z\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.619582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.619500    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:19.924918 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.924881    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"]
Oct 31 00:45:19.925620 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:19.925594    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d895f4e_1d56_41ff_b99a_3959a00bc5e9.slice/crio-a6d03032c9f6288954a149bd3d3e2a5e258dbaa1a1256df4cbd16e77742de84d WatchSource:0}: Error finding container a6d03032c9f6288954a149bd3d3e2a5e258dbaa1a1256df4cbd16e77742de84d: Status 404 returned error can't find the container with id a6d03032c9f6288954a149bd3d3e2a5e258dbaa1a1256df4cbd16e77742de84d
Oct 31 00:45:19.988381 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.988352    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"
Oct 31 00:45:19.988523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.988382    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl" event={"ID":"139c87ed-b8ff-484c-b471-5684a5023a5b","Type":"ContainerDied","Data":"7687b2fae147da716b6845169a510f64e2ab6959b5309b3f8c43da1159ac9d87"}
Oct 31 00:45:19.988523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.988434    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7687b2fae147da716b6845169a510f64e2ab6959b5309b3f8c43da1159ac9d87"
Oct 31 00:45:19.989778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:19.989739    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z" event={"ID":"9d895f4e-1d56-41ff-b99a-3959a00bc5e9","Type":"ContainerStarted","Data":"a6d03032c9f6288954a149bd3d3e2a5e258dbaa1a1256df4cbd16e77742de84d"}
Oct 31 00:45:21.000572 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:21.000529    4924 generic.go:356] "Generic (PLEG): container finished" podID="9d895f4e-1d56-41ff-b99a-3959a00bc5e9" containerID="a945447e3d4a60996911c04663d848857a138cfcace2fde103aee8ad3271407e" exitCode=0
Oct 31 00:45:21.001118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:21.000589    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z" event={"ID":"9d895f4e-1d56-41ff-b99a-3959a00bc5e9","Type":"ContainerDied","Data":"a945447e3d4a60996911c04663d848857a138cfcace2fde103aee8ad3271407e"}
Oct 31 00:45:21.757318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:21.757287    4924 scope.go:122] "RemoveContainer" containerID="640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff"
Oct 31 00:45:22.174395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.174368    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:22.325982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.325896    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca\") pod \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") "
Oct 31 00:45:22.325982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.325969    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg\" (UniqueName: \"kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg\") pod \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") "
Oct 31 00:45:22.326182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.325990    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca\") pod \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\" (UID: \"9d895f4e-1d56-41ff-b99a-3959a00bc5e9\") "
Oct 31 00:45:22.326658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.326625    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca" pod "9d895f4e-1d56-41ff-b99a-3959a00bc5e9" (UID: "9d895f4e-1d56-41ff-b99a-3959a00bc5e9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:22.328338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.328312    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg" pod "9d895f4e-1d56-41ff-b99a-3959a00bc5e9" (UID: "9d895f4e-1d56-41ff-b99a-3959a00bc5e9"). InnerVolumeSpecName "kube-api-access-hfxcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:22.328556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.328541    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca" pod "9d895f4e-1d56-41ff-b99a-3959a00bc5e9" (UID: "9d895f4e-1d56-41ff-b99a-3959a00bc5e9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:22.427942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.427907    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfxcg\" (UniqueName: \"kubernetes.io/projected/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-kube-api-access-hfxcg\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:22.427942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.427937    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:22.427942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.427948    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9d895f4e-1d56-41ff-b99a-3959a00bc5e9-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:22.756782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:22.756726    4924 scope.go:122] "RemoveContainer" containerID="5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442"
Oct 31 00:45:23.020431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.020398    4924 generic.go:356] "Generic (PLEG): container finished" podID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerID="e17659172066b20acbe4a406a5e0ec868296e698d51b29ef90d8994ce7c15157" exitCode=60
Oct 31 00:45:23.020616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.020475    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerDied","Data":"e17659172066b20acbe4a406a5e0ec868296e698d51b29ef90d8994ce7c15157"}
Oct 31 00:45:23.020616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.020580    4924 scope.go:122] "RemoveContainer" containerID="640b9538e4be88854ec7882b7c6bacdbb88a74c99a68543bc9ac251ca3826bff"
Oct 31 00:45:23.021042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.021021    4924 scope.go:122] "RemoveContainer" containerID="e17659172066b20acbe4a406a5e0ec868296e698d51b29ef90d8994ce7c15157"
Oct 31 00:45:23.021385 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:23.021359    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890125-h55sd_odh-ai-gateway-infra(5f6a71e9-a9e4-4071-b335-4bf48f2e3a23)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23"
Oct 31 00:45:23.022111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.022085    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"
Oct 31 00:45:23.022218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.022119    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z" event={"ID":"9d895f4e-1d56-41ff-b99a-3959a00bc5e9","Type":"ContainerDied","Data":"a6d03032c9f6288954a149bd3d3e2a5e258dbaa1a1256df4cbd16e77742de84d"}
Oct 31 00:45:23.022218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.022150    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6d03032c9f6288954a149bd3d3e2a5e258dbaa1a1256df4cbd16e77742de84d"
Oct 31 00:45:23.757094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:23.757061    4924 scope.go:122] "RemoveContainer" containerID="0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4"
Oct 31 00:45:24.031359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.031316    4924 generic.go:356] "Generic (PLEG): container finished" podID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerID="2e5135adc9e13f8e9ef8af2f2dcdd053d36372f46bea77b14e58478aaaf28b96" exitCode=60
Oct 31 00:45:24.031508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.031391    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerDied","Data":"2e5135adc9e13f8e9ef8af2f2dcdd053d36372f46bea77b14e58478aaaf28b96"}
Oct 31 00:45:24.031508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.031460    4924 scope.go:122] "RemoveContainer" containerID="5a3dc389162ee8adc777d22c9a45a3d9b3be15a527aefa3eb077c20b224d0442"
Oct 31 00:45:24.031920 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.031902    4924 scope.go:122] "RemoveContainer" containerID="2e5135adc9e13f8e9ef8af2f2dcdd053d36372f46bea77b14e58478aaaf28b96"
Oct 31 00:45:24.032254 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:24.032229    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6_odh-ai-gateway-infra(359b6fb8-0144-4830-9171-21e5a3d888cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb"
Oct 31 00:45:24.050161 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.050135    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"]
Oct 31 00:45:24.235455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.235425    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:24.383086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.382983    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca\") pod \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") "
Oct 31 00:45:24.383086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.383067    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p\" (UniqueName: \"kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p\") pod \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\" (UID: \"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23\") "
Oct 31 00:45:24.383472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.383444    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca" pod "5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" (UID: "5f6a71e9-a9e4-4071-b335-4bf48f2e3a23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:24.385384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.385357    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p" pod "5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" (UID: "5f6a71e9-a9e4-4071-b335-4bf48f2e3a23"). InnerVolumeSpecName "kube-api-access-dmb9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:24.484428 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.484391    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-maas-api-service-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:24.484428 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.484422    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmb9p\" (UniqueName: \"kubernetes.io/projected/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23-kube-api-access-dmb9p\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:24.757841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.757471    4924 scope.go:122] "RemoveContainer" containerID="7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd"
Oct 31 00:45:24.804773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:24.787098    4924 scope.go:122] "RemoveContainer" containerID="c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9"
Oct 31 00:45:25.078771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.078223    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"]
Oct 31 00:45:25.237771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.225993    4924 generic.go:356] "Generic (PLEG): container finished" podID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerID="5ceb1fe8361219026e01b7f25fffa5476e454255fad71e8a34a50fbbe592ba7a" exitCode=60
Oct 31 00:45:25.237771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.226176    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerDied","Data":"5ceb1fe8361219026e01b7f25fffa5476e454255fad71e8a34a50fbbe592ba7a"}
Oct 31 00:45:25.237771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.226242    4924 scope.go:122] "RemoveContainer" containerID="0b63789cab5e8d020131c02282d6e4a30365eb14ab2bbda91ffb86f823344ba4"
Oct 31 00:45:25.237771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.226716    4924 scope.go:122] "RemoveContainer" containerID="5ceb1fe8361219026e01b7f25fffa5476e454255fad71e8a34a50fbbe592ba7a"
Oct 31 00:45:25.252768 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:25.250029    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6_odh-ai-gateway-infra(d7c8d87f-5619-4988-a7d2-ea084e5b3970)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970"
Oct 31 00:45:25.263785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.263497    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd" event={"ID":"5f6a71e9-a9e4-4071-b335-4bf48f2e3a23","Type":"ContainerDied","Data":"58b4c85755f02ee9336cca9d93af89ffc26b0d8ad3bac37bb4de5f1d3fca71e5"}
Oct 31 00:45:25.263785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.263634    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"
Oct 31 00:45:25.309024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.308986    4924 scope.go:122] "RemoveContainer" containerID="e17659172066b20acbe4a406a5e0ec868296e698d51b29ef90d8994ce7c15157"
Oct 31 00:45:25.399187 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.399154    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"]
Oct 31 00:45:25.445500 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.444176    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890125-h55sd"]
Oct 31 00:45:25.985278 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.985245    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" path="/var/lib/kubelet/pods/5f6a71e9-a9e4-4071-b335-4bf48f2e3a23/volumes"
Oct 31 00:45:25.989057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:25.989027    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:26.077498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.077405    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc\" (UniqueName: \"kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc\") pod \"359b6fb8-0144-4830-9171-21e5a3d888cb\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") "
Oct 31 00:45:26.077688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.077563    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca\") pod \"359b6fb8-0144-4830-9171-21e5a3d888cb\" (UID: \"359b6fb8-0144-4830-9171-21e5a3d888cb\") "
Oct 31 00:45:26.078148 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.078116    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca" pod "359b6fb8-0144-4830-9171-21e5a3d888cb" (UID: "359b6fb8-0144-4830-9171-21e5a3d888cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:26.079859 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.079836    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc" pod "359b6fb8-0144-4830-9171-21e5a3d888cb" (UID: "359b6fb8-0144-4830-9171-21e5a3d888cb"). InnerVolumeSpecName "kube-api-access-n97fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:26.178907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.178868    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/359b6fb8-0144-4830-9171-21e5a3d888cb-maas-api-service-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:26.178907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.178904    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n97fc\" (UniqueName: \"kubernetes.io/projected/359b6fb8-0144-4830-9171-21e5a3d888cb-kube-api-access-n97fc\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:26.275233 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.275201    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"
Oct 31 00:45:26.275418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.275228    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6" event={"ID":"359b6fb8-0144-4830-9171-21e5a3d888cb","Type":"ContainerDied","Data":"daa2adbcf6ee20150cd145c7e40ed132409875b8106cc45c2b6755b9f3703db2"}
Oct 31 00:45:26.275418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.275305    4924 scope.go:122] "RemoveContainer" containerID="2e5135adc9e13f8e9ef8af2f2dcdd053d36372f46bea77b14e58478aaaf28b96"
Oct 31 00:45:26.289725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.289694    4924 generic.go:356] "Generic (PLEG): container finished" podID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerID="450502ef853d8f37f857aaedc4d5427194eda0f9c6c7d7c569e88ac1395fcd66" exitCode=60
Oct 31 00:45:26.289910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.289784    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerDied","Data":"450502ef853d8f37f857aaedc4d5427194eda0f9c6c7d7c569e88ac1395fcd66"}
Oct 31 00:45:26.290628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.290228    4924 scope.go:122] "RemoveContainer" containerID="450502ef853d8f37f857aaedc4d5427194eda0f9c6c7d7c569e88ac1395fcd66"
Oct 31 00:45:26.290628 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:26.290495    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8_odh-ai-gateway-infra(9c2867d9-b020-4a26-b76c-6fb89d073901)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901"
Oct 31 00:45:26.290871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.290851    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"]
Oct 31 00:45:26.367783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.367701    4924 scope.go:122] "RemoveContainer" containerID="c7b81ef3ccf5d580a516f909ee7edb87ad5d204f0f7da81759dba662b1f741a9"
Oct 31 00:45:26.375334 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.374784    4924 generic.go:356] "Generic (PLEG): container finished" podID="f68da95a-8b68-4c68-a555-308b36413d92" containerID="ae3517a3cdba0f35927750a7b88c26b0fc3bec2953ca2abf7b883d16126b5e26" exitCode=60
Oct 31 00:45:26.375334 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.374876    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" event={"ID":"f68da95a-8b68-4c68-a555-308b36413d92","Type":"ContainerDied","Data":"ae3517a3cdba0f35927750a7b88c26b0fc3bec2953ca2abf7b883d16126b5e26"}
Oct 31 00:45:26.377697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.375758    4924 scope.go:122] "RemoveContainer" containerID="ae3517a3cdba0f35927750a7b88c26b0fc3bec2953ca2abf7b883d16126b5e26"
Oct 31 00:45:26.377697 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:26.376114    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj_odh-ai-gateway-infra(f68da95a-8b68-4c68-a555-308b36413d92)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" podUID="f68da95a-8b68-4c68-a555-308b36413d92"
Oct 31 00:45:26.384165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.384137    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"]
Oct 31 00:45:26.390102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.390060    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-5c2a03-29890125-dtll6"]
Oct 31 00:45:26.396919 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.396891    4924 scope.go:122] "RemoveContainer" containerID="7efd5dac1f33ed1f17697f0106ec2e3231c0e79211286e5d8f74ec9837489bdd"
Oct 31 00:45:26.685913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.685890    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:26.808384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.803292    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca\") pod \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") "
Oct 31 00:45:26.808384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.803357    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4\" (UniqueName: \"kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4\") pod \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\" (UID: \"d7c8d87f-5619-4988-a7d2-ea084e5b3970\") "
Oct 31 00:45:26.808384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.806409    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca" pod "d7c8d87f-5619-4988-a7d2-ea084e5b3970" (UID: "d7c8d87f-5619-4988-a7d2-ea084e5b3970"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:26.826347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.826316    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4" pod "d7c8d87f-5619-4988-a7d2-ea084e5b3970" (UID: "d7c8d87f-5619-4988-a7d2-ea084e5b3970"). InnerVolumeSpecName "kube-api-access-j5xf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:26.929823 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.928577    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8d87f-5619-4988-a7d2-ea084e5b3970-maas-api-service-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:26.929823 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:26.928615    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5xf4\" (UniqueName: \"kubernetes.io/projected/d7c8d87f-5619-4988-a7d2-ea084e5b3970-kube-api-access-j5xf4\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:27.346024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.345024    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"]
Oct 31 00:45:27.382161 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.382122    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6" event={"ID":"d7c8d87f-5619-4988-a7d2-ea084e5b3970","Type":"ContainerDied","Data":"a82496fa07d5e9a96ea1874404a42443f4a9e24dd5350a370af7583629dc322f"}
Oct 31 00:45:27.382311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.382178    4924 scope.go:122] "RemoveContainer" containerID="5ceb1fe8361219026e01b7f25fffa5476e454255fad71e8a34a50fbbe592ba7a"
Oct 31 00:45:27.382311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.382130    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"
Oct 31 00:45:27.466698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.462361    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"]
Oct 31 00:45:27.482963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.482877    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"]
Oct 31 00:45:27.542868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.542832    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-44bc67-29890125-zx7j6"]
Oct 31 00:45:27.893560 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.893525    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" path="/var/lib/kubelet/pods/359b6fb8-0144-4830-9171-21e5a3d888cb/volumes"
Oct 31 00:45:27.894386 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:27.894359    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" path="/var/lib/kubelet/pods/d7c8d87f-5619-4988-a7d2-ea084e5b3970/volumes"
Oct 31 00:45:28.001999 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:28.001940    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:45:28.009284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.009257    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:28.056796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.052606    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:28.098087 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.098046    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj\" (UniqueName: \"kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj\") pod \"f68da95a-8b68-4c68-a555-308b36413d92\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") "
Oct 31 00:45:28.098374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.098352    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca\") pod \"9c2867d9-b020-4a26-b76c-6fb89d073901\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") "
Oct 31 00:45:28.098564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.098546    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca\") pod \"f68da95a-8b68-4c68-a555-308b36413d92\" (UID: \"f68da95a-8b68-4c68-a555-308b36413d92\") "
Oct 31 00:45:28.099840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.099812    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca" pod "f68da95a-8b68-4c68-a555-308b36413d92" (UID: "f68da95a-8b68-4c68-a555-308b36413d92"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:28.100665 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.100641    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca" pod "9c2867d9-b020-4a26-b76c-6fb89d073901" (UID: "9c2867d9-b020-4a26-b76c-6fb89d073901"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:28.104080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.104051    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj" pod "f68da95a-8b68-4c68-a555-308b36413d92" (UID: "f68da95a-8b68-4c68-a555-308b36413d92"). InnerVolumeSpecName "kube-api-access-x96sj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:28.200610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.200520    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9\" (UniqueName: \"kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9\") pod \"9c2867d9-b020-4a26-b76c-6fb89d073901\" (UID: \"9c2867d9-b020-4a26-b76c-6fb89d073901\") "
Oct 31 00:45:28.201314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.201288    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2867d9-b020-4a26-b76c-6fb89d073901-maas-api-service-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:28.201421 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.201409    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68da95a-8b68-4c68-a555-308b36413d92-maas-api-service-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:28.201501 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.201490    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x96sj\" (UniqueName: \"kubernetes.io/projected/f68da95a-8b68-4c68-a555-308b36413d92-kube-api-access-x96sj\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:28.206122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.206094    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9" pod "9c2867d9-b020-4a26-b76c-6fb89d073901" (UID: "9c2867d9-b020-4a26-b76c-6fb89d073901"). InnerVolumeSpecName "kube-api-access-92rz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:28.304861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.304810    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92rz9\" (UniqueName: \"kubernetes.io/projected/9c2867d9-b020-4a26-b76c-6fb89d073901-kube-api-access-92rz9\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:28.437903 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.437861    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8" event={"ID":"9c2867d9-b020-4a26-b76c-6fb89d073901","Type":"ContainerDied","Data":"54078018532ed162f691627426a58ae472be1713b836ff728b3c61bf7cf91cf1"}
Oct 31 00:45:28.438507 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.438487    4924 scope.go:122] "RemoveContainer" containerID="450502ef853d8f37f857aaedc4d5427194eda0f9c6c7d7c569e88ac1395fcd66"
Oct 31 00:45:28.438775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.438735    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"
Oct 31 00:45:28.464678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.464651    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj" event={"ID":"f68da95a-8b68-4c68-a555-308b36413d92","Type":"ContainerDied","Data":"1373c1049ce7bcdf06696d11ae0d66bac8d366819be41e7b57b42e06a606aaa4"}
Oct 31 00:45:28.464918 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.464899    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"
Oct 31 00:45:28.493766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.487454    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"]
Oct 31 00:45:28.505210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.504253    4924 scope.go:122] "RemoveContainer" containerID="ae3517a3cdba0f35927750a7b88c26b0fc3bec2953ca2abf7b883d16126b5e26"
Oct 31 00:45:28.511788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.510648    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f99ac0-29890125-nhbx8"]
Oct 31 00:45:28.547766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.545566    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"]
Oct 31 00:45:28.551767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.551213    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b433e6-29890125-pw6dj"]
Oct 31 00:45:28.965045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965007    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"]
Oct 31 00:45:28.965582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965560    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.965582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965580    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965590    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965595    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965610    4924 cpu_manager.go:414] "containerMap: removing container" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965615    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965624    4924 cpu_manager.go:414] "containerMap: removing container" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965632    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965645    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965650    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965662    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965667    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965675    4924 cpu_manager.go:414] "containerMap: removing container" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965679    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965686    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965691    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965702    4924 cpu_manager.go:414] "containerMap: removing container" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965709    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965728    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965734    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965755    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9d895f4e-1d56-41ff-b99a-3959a00bc5e9" containerName="revoke-keys"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965763    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d895f4e-1d56-41ff-b99a-3959a00bc5e9" containerName="revoke-keys"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965779    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965786    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965799    4924 cpu_manager.go:414] "containerMap: removing container" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965806    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965818    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.965816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965822    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965920    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965929    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965936    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965944    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965949    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965959    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965968    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965973    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965980    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965984    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="f68da95a-8b68-4c68-a555-308b36413d92" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965991    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.965997    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:28.966968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.966003    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d895f4e-1d56-41ff-b99a-3959a00bc5e9" containerName="revoke-keys"
Oct 31 00:45:28.970993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.970970    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:28.977049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.977022    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:45:28.985021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.981159    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:45:28.985147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:28.985130    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"]
Oct 31 00:45:29.030803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.030483    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdp8v\" (UniqueName: \"kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.030803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.030704    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.030803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.030737    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.131626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.131589    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.131626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.131635    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.131901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.131730    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdp8v\" (UniqueName: \"kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.132500 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.132469    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.137710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.137636    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.140145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.140117    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdp8v\" (UniqueName: \"kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v\") pod \"maas-api-revoke-sub-ec21c7ce-bxdkn\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.303726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.303550    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:29.594444 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.594407    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"]
Oct 31 00:45:29.595490 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:29.595459    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaeee1db_8c68_443f_90c3_3be9702a6b8d.slice/crio-a263659c89e3ac7c2b963c20e9d5bea4d672431c801ff7a6775b683174f58ed3 WatchSource:0}: Error finding container a263659c89e3ac7c2b963c20e9d5bea4d672431c801ff7a6775b683174f58ed3: Status 404 returned error can't find the container with id a263659c89e3ac7c2b963c20e9d5bea4d672431c801ff7a6775b683174f58ed3
Oct 31 00:45:29.766416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.766378    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" path="/var/lib/kubelet/pods/9c2867d9-b020-4a26-b76c-6fb89d073901/volumes"
Oct 31 00:45:29.766822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.766808    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f68da95a-8b68-4c68-a555-308b36413d92" path="/var/lib/kubelet/pods/f68da95a-8b68-4c68-a555-308b36413d92/volumes"
Oct 31 00:45:29.772536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.772511    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"]
Oct 31 00:45:29.773026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773014    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:29.773065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773028    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c8d87f-5619-4988-a7d2-ea084e5b3970" containerName="cleanup"
Oct 31 00:45:29.773065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773043    4924 cpu_manager.go:414] "containerMap: removing container" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:29.773065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773048    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:29.773172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773152    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f6a71e9-a9e4-4071-b335-4bf48f2e3a23" containerName="cleanup"
Oct 31 00:45:29.773172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773159    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="359b6fb8-0144-4830-9171-21e5a3d888cb" containerName="cleanup"
Oct 31 00:45:29.773172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.773169    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c2867d9-b020-4a26-b76c-6fb89d073901" containerName="cleanup"
Oct 31 00:45:29.777944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.777912    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.785607 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.785584    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"]
Oct 31 00:45:29.849094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.849011    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkd7x\" (UniqueName: \"kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.849094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.849081    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.849323 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.849280    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.950090 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.950047    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkd7x\" (UniqueName: \"kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.950381 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.950118    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.950381 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.950284    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.951007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.950982    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.952618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.952598    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:29.959001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:29.958974    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkd7x\" (UniqueName: \"kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x\") pod \"maas-api-revoke-sub-d5cc4c7b-q62tn\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:30.095850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.095806    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:30.262216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.262182    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"]
Oct 31 00:45:30.267308 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.267290    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.272813 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.272786    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"]
Oct 31 00:45:30.354381 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.354350    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtjmp\" (UniqueName: \"kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.354381 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.354392    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.354597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.354537    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.399477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.399430    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"]
Oct 31 00:45:30.405740 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:30.405703    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb2bc97_f1a9_4efa_b0cf_b85f4388f6f7.slice/crio-9d45a259eaaa5d7a3489c84e84f6db5c9b187a67316724611bb6f4ccd455cca9 WatchSource:0}: Error finding container 9d45a259eaaa5d7a3489c84e84f6db5c9b187a67316724611bb6f4ccd455cca9: Status 404 returned error can't find the container with id 9d45a259eaaa5d7a3489c84e84f6db5c9b187a67316724611bb6f4ccd455cca9
Oct 31 00:45:30.455423 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.455395    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtjmp\" (UniqueName: \"kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.455539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.455437    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.455584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.455551    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.456426 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.456405    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.457852 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.457833    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.470078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.470055    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtjmp\" (UniqueName: \"kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp\") pod \"maas-api-revoke-sub-3f81abd9-22mn5\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.482216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.482183    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn" event={"ID":"baeee1db-8c68-443f-90c3-3be9702a6b8d","Type":"ContainerStarted","Data":"479ea9508faca66d776971888731df71e808365664e5b2fcd89bf8f443a20093"}
Oct 31 00:45:30.482312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.482240    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn" event={"ID":"baeee1db-8c68-443f-90c3-3be9702a6b8d","Type":"ContainerStarted","Data":"a263659c89e3ac7c2b963c20e9d5bea4d672431c801ff7a6775b683174f58ed3"}
Oct 31 00:45:30.483520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.483500    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn" event={"ID":"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7","Type":"ContainerStarted","Data":"9d45a259eaaa5d7a3489c84e84f6db5c9b187a67316724611bb6f4ccd455cca9"}
Oct 31 00:45:30.519250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.519202    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn" podStartSLOduration=2.519189013 podStartE2EDuration="2.519189013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:30.516482673 +0000 UTC m=+730.887266462" watchObservedRunningTime="2026-10-31 00:45:30.519189013 +0000 UTC m=+730.889972828"
Oct 31 00:45:30.585378 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.585349    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:30.872142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:30.872109    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"]
Oct 31 00:45:30.877023 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:30.876997    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0bb425_505d_4ba2_80e8_b256b559b46e.slice/crio-10ec3ac1c74a878d53849c9ed96f8e9ba16e01870e065d4b874bab02331d56a6 WatchSource:0}: Error finding container 10ec3ac1c74a878d53849c9ed96f8e9ba16e01870e065d4b874bab02331d56a6: Status 404 returned error can't find the container with id 10ec3ac1c74a878d53849c9ed96f8e9ba16e01870e065d4b874bab02331d56a6
Oct 31 00:45:31.492098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.492013    4924 generic.go:356] "Generic (PLEG): container finished" podID="baeee1db-8c68-443f-90c3-3be9702a6b8d" containerID="479ea9508faca66d776971888731df71e808365664e5b2fcd89bf8f443a20093" exitCode=0
Oct 31 00:45:31.492245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.492093    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn" event={"ID":"baeee1db-8c68-443f-90c3-3be9702a6b8d","Type":"ContainerDied","Data":"479ea9508faca66d776971888731df71e808365664e5b2fcd89bf8f443a20093"}
Oct 31 00:45:31.493880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.493857    4924 generic.go:356] "Generic (PLEG): container finished" podID="ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" containerID="c616192ec1f71c81c2fddf238e61e2871fd57bf17e5611dc6f8590b1e5581ec6" exitCode=0
Oct 31 00:45:31.494018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.493921    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn" event={"ID":"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7","Type":"ContainerDied","Data":"c616192ec1f71c81c2fddf238e61e2871fd57bf17e5611dc6f8590b1e5581ec6"}
Oct 31 00:45:31.495675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.495654    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5" event={"ID":"9f0bb425-505d-4ba2-80e8-b256b559b46e","Type":"ContainerStarted","Data":"dbba89321e9b73943b74625fd28e02f2c7fe5beabf60fadfd30208ddd6a4b134"}
Oct 31 00:45:31.495777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.495691    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5" event={"ID":"9f0bb425-505d-4ba2-80e8-b256b559b46e","Type":"ContainerStarted","Data":"10ec3ac1c74a878d53849c9ed96f8e9ba16e01870e065d4b874bab02331d56a6"}
Oct 31 00:45:31.543550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:31.543497    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5" podStartSLOduration=1.543483155 podStartE2EDuration="1.543483155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:31.541393039 +0000 UTC m=+731.912176828" watchObservedRunningTime="2026-10-31 00:45:31.543483155 +0000 UTC m=+731.914266944"
Oct 31 00:45:32.505099 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.505066    4924 generic.go:356] "Generic (PLEG): container finished" podID="9f0bb425-505d-4ba2-80e8-b256b559b46e" containerID="dbba89321e9b73943b74625fd28e02f2c7fe5beabf60fadfd30208ddd6a4b134" exitCode=0
Oct 31 00:45:32.505551 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.505160    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5" event={"ID":"9f0bb425-505d-4ba2-80e8-b256b559b46e","Type":"ContainerDied","Data":"dbba89321e9b73943b74625fd28e02f2c7fe5beabf60fadfd30208ddd6a4b134"}
Oct 31 00:45:32.673429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.673401    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:32.763934 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.763869    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:32.780266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.780239    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca\") pod \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") "
Oct 31 00:45:32.780384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.780299    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca\") pod \"baeee1db-8c68-443f-90c3-3be9702a6b8d\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") "
Oct 31 00:45:32.780481 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.780462    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca\") pod \"baeee1db-8c68-443f-90c3-3be9702a6b8d\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") "
Oct 31 00:45:32.780584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.780566    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca\") pod \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") "
Oct 31 00:45:32.780645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.780611    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca" pod "ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" (UID: "ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:32.780895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.780872    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca" pod "baeee1db-8c68-443f-90c3-3be9702a6b8d" (UID: "baeee1db-8c68-443f-90c3-3be9702a6b8d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:32.781045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.781022    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v\" (UniqueName: \"kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v\") pod \"baeee1db-8c68-443f-90c3-3be9702a6b8d\" (UID: \"baeee1db-8c68-443f-90c3-3be9702a6b8d\") "
Oct 31 00:45:32.781198 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.781179    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x\" (UniqueName: \"kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x\") pod \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\" (UID: \"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7\") "
Oct 31 00:45:32.783418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.783387    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca" pod "ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" (UID: "ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:32.784336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.784304    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x" pod "ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" (UID: "ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7"). InnerVolumeSpecName "kube-api-access-kkd7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787108    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca" pod "baeee1db-8c68-443f-90c3-3be9702a6b8d" (UID: "baeee1db-8c68-443f-90c3-3be9702a6b8d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787519    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v" pod "baeee1db-8c68-443f-90c3-3be9702a6b8d" (UID: "baeee1db-8c68-443f-90c3-3be9702a6b8d"). InnerVolumeSpecName "kube-api-access-qdp8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787632    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787792    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787814    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/baeee1db-8c68-443f-90c3-3be9702a6b8d-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787828    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787842    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdp8v\" (UniqueName: \"kubernetes.io/projected/baeee1db-8c68-443f-90c3-3be9702a6b8d-kube-api-access-qdp8v\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:32.792489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:32.787856    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkd7x\" (UniqueName: \"kubernetes.io/projected/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7-kube-api-access-kkd7x\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:33.204587 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.204552    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"]
Oct 31 00:45:33.205135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.205087    4924 cpu_manager.go:414] "containerMap: removing container" podUID="baeee1db-8c68-443f-90c3-3be9702a6b8d" containerName="revoke-keys"
Oct 31 00:45:33.205135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.205102    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baeee1db-8c68-443f-90c3-3be9702a6b8d" containerName="revoke-keys"
Oct 31 00:45:33.205239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.205138    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" containerName="revoke-keys"
Oct 31 00:45:33.205239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.205144    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" containerName="revoke-keys"
Oct 31 00:45:33.205239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.205234    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="baeee1db-8c68-443f-90c3-3be9702a6b8d" containerName="revoke-keys"
Oct 31 00:45:33.205379 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.205253    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" containerName="revoke-keys"
Oct 31 00:45:33.212733 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.212710    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.235956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.235923    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"]
Oct 31 00:45:33.293210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.293171    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.293210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.293209    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcftj\" (UniqueName: \"kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.293454 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.293229    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.394547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.394512    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.394696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.394551    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcftj\" (UniqueName: \"kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.394696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.394582    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.395503 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.395477    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.396944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.396920    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.402875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.402852    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcftj\" (UniqueName: \"kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj\") pod \"maas-api-revoke-sub-e97ee4ae-hb9md\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.514260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.514178    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"
Oct 31 00:45:33.514260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.514194    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn" event={"ID":"baeee1db-8c68-443f-90c3-3be9702a6b8d","Type":"ContainerDied","Data":"a263659c89e3ac7c2b963c20e9d5bea4d672431c801ff7a6775b683174f58ed3"}
Oct 31 00:45:33.514260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.514258    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a263659c89e3ac7c2b963c20e9d5bea4d672431c801ff7a6775b683174f58ed3"
Oct 31 00:45:33.516042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.516021    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"
Oct 31 00:45:33.516042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.516029    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn" event={"ID":"ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7","Type":"ContainerDied","Data":"9d45a259eaaa5d7a3489c84e84f6db5c9b187a67316724611bb6f4ccd455cca9"}
Oct 31 00:45:33.516226 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.516080    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d45a259eaaa5d7a3489c84e84f6db5c9b187a67316724611bb6f4ccd455cca9"
Oct 31 00:45:33.547788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.547759    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:33.743288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.743262    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:33.801113 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.800911    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp\" (UniqueName: \"kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp\") pod \"9f0bb425-505d-4ba2-80e8-b256b559b46e\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") "
Oct 31 00:45:33.801113 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.801090    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca\") pod \"9f0bb425-505d-4ba2-80e8-b256b559b46e\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") "
Oct 31 00:45:33.801358 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.801247    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca\") pod \"9f0bb425-505d-4ba2-80e8-b256b559b46e\" (UID: \"9f0bb425-505d-4ba2-80e8-b256b559b46e\") "
Oct 31 00:45:33.802299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.802267    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca" pod "9f0bb425-505d-4ba2-80e8-b256b559b46e" (UID: "9f0bb425-505d-4ba2-80e8-b256b559b46e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:33.803440 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.803415    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca" pod "9f0bb425-505d-4ba2-80e8-b256b559b46e" (UID: "9f0bb425-505d-4ba2-80e8-b256b559b46e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:33.803531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.803511    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp" pod "9f0bb425-505d-4ba2-80e8-b256b559b46e" (UID: "9f0bb425-505d-4ba2-80e8-b256b559b46e"). InnerVolumeSpecName "kube-api-access-gtjmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:33.845721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.845697    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"]
Oct 31 00:45:33.845977 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:33.845953    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32833fc2_4d9f_4731_bed2_8112b9bc67ff.slice/crio-271ca6cde96ab21a895ba5c4264c843c758b79a36959158ff63c287864e1b9b8 WatchSource:0}: Error finding container 271ca6cde96ab21a895ba5c4264c843c758b79a36959158ff63c287864e1b9b8: Status 404 returned error can't find the container with id 271ca6cde96ab21a895ba5c4264c843c758b79a36959158ff63c287864e1b9b8
Oct 31 00:45:33.903337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.903300    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:33.903337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.903340    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtjmp\" (UniqueName: \"kubernetes.io/projected/9f0bb425-505d-4ba2-80e8-b256b559b46e-kube-api-access-gtjmp\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:33.903517 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:33.903355    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f0bb425-505d-4ba2-80e8-b256b559b46e-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:34.528695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:34.528657    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md" event={"ID":"32833fc2-4d9f-4731-bed2-8112b9bc67ff","Type":"ContainerStarted","Data":"7afa6c0977e64891cc1b1a2d5ccd9bec54798381c3756553c3d8eab471e1c733"}
Oct 31 00:45:34.529305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:34.529280    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md" event={"ID":"32833fc2-4d9f-4731-bed2-8112b9bc67ff","Type":"ContainerStarted","Data":"271ca6cde96ab21a895ba5c4264c843c758b79a36959158ff63c287864e1b9b8"}
Oct 31 00:45:34.531328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:34.531299    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5" event={"ID":"9f0bb425-505d-4ba2-80e8-b256b559b46e","Type":"ContainerDied","Data":"10ec3ac1c74a878d53849c9ed96f8e9ba16e01870e065d4b874bab02331d56a6"}
Oct 31 00:45:34.531488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:34.531474    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10ec3ac1c74a878d53849c9ed96f8e9ba16e01870e065d4b874bab02331d56a6"
Oct 31 00:45:34.531668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:34.531643    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"
Oct 31 00:45:34.546958 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:34.546911    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md" podStartSLOduration=1.546896365 podStartE2EDuration="1.546896365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:34.546092981 +0000 UTC m=+734.916876771" watchObservedRunningTime="2026-10-31 00:45:34.546896365 +0000 UTC m=+734.917680195"
Oct 31 00:45:35.540854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:35.540821    4924 generic.go:356] "Generic (PLEG): container finished" podID="32833fc2-4d9f-4731-bed2-8112b9bc67ff" containerID="7afa6c0977e64891cc1b1a2d5ccd9bec54798381c3756553c3d8eab471e1c733" exitCode=0
Oct 31 00:45:35.541279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:35.540918    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md" event={"ID":"32833fc2-4d9f-4731-bed2-8112b9bc67ff","Type":"ContainerDied","Data":"7afa6c0977e64891cc1b1a2d5ccd9bec54798381c3756553c3d8eab471e1c733"}
Oct 31 00:45:36.715100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.715076    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:36.850098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.850010    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca\") pod \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") "
Oct 31 00:45:36.850316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.850295    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj\" (UniqueName: \"kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj\") pod \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") "
Oct 31 00:45:36.850388 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.850338    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca\") pod \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\" (UID: \"32833fc2-4d9f-4731-bed2-8112b9bc67ff\") "
Oct 31 00:45:36.850388 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.850365    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca" pod "32833fc2-4d9f-4731-bed2-8112b9bc67ff" (UID: "32833fc2-4d9f-4731-bed2-8112b9bc67ff"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:36.850880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.850842    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:36.852552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.852528    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca" pod "32833fc2-4d9f-4731-bed2-8112b9bc67ff" (UID: "32833fc2-4d9f-4731-bed2-8112b9bc67ff"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:36.852658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.852594    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj" pod "32833fc2-4d9f-4731-bed2-8112b9bc67ff" (UID: "32833fc2-4d9f-4731-bed2-8112b9bc67ff"). InnerVolumeSpecName "kube-api-access-jcftj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:36.952223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.952184    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcftj\" (UniqueName: \"kubernetes.io/projected/32833fc2-4d9f-4731-bed2-8112b9bc67ff-kube-api-access-jcftj\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:36.952223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:36.952224    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32833fc2-4d9f-4731-bed2-8112b9bc67ff-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:37.559637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:37.559604    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"
Oct 31 00:45:37.559845 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:37.559645    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md" event={"ID":"32833fc2-4d9f-4731-bed2-8112b9bc67ff","Type":"ContainerDied","Data":"271ca6cde96ab21a895ba5c4264c843c758b79a36959158ff63c287864e1b9b8"}
Oct 31 00:45:37.559845 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:37.559701    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="271ca6cde96ab21a895ba5c4264c843c758b79a36959158ff63c287864e1b9b8"
Oct 31 00:45:39.983790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.983738    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"]
Oct 31 00:45:39.984291 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.984277    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9f0bb425-505d-4ba2-80e8-b256b559b46e" containerName="revoke-keys"
Oct 31 00:45:39.984336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.984292    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f0bb425-505d-4ba2-80e8-b256b559b46e" containerName="revoke-keys"
Oct 31 00:45:39.984336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.984329    4924 cpu_manager.go:414] "containerMap: removing container" podUID="32833fc2-4d9f-4731-bed2-8112b9bc67ff" containerName="revoke-keys"
Oct 31 00:45:39.984336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.984334    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32833fc2-4d9f-4731-bed2-8112b9bc67ff" containerName="revoke-keys"
Oct 31 00:45:39.984429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.984418    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="32833fc2-4d9f-4731-bed2-8112b9bc67ff" containerName="revoke-keys"
Oct 31 00:45:39.984462 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.984433    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f0bb425-505d-4ba2-80e8-b256b559b46e" containerName="revoke-keys"
Oct 31 00:45:39.990827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.990812    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.010766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.995119    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:45:40.010766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:39.995173    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:45:40.014818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.014789    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"]
Oct 31 00:45:40.052629 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.052588    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.052852 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.052828    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmp9d\" (UniqueName: \"kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.053047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.052891    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.153702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.153655    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmp9d\" (UniqueName: \"kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.153960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.153721    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.153960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.153780    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.154543 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.154511    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.157125 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.157104    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.163565 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.163538    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmp9d\" (UniqueName: \"kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d\") pod \"maas-api-revoke-sub-47059f59-vpzxc\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.333065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.333011    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:40.619916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.619887    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"]
Oct 31 00:45:40.620980 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:40.620955    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a3cf8a_e155_4b83_9b09_4983a0bda04f.slice/crio-651a95bc6b9a4d73d4acee85d45078582642dd17e2c36e9f40a87631160c464d WatchSource:0}: Error finding container 651a95bc6b9a4d73d4acee85d45078582642dd17e2c36e9f40a87631160c464d: Status 404 returned error can't find the container with id 651a95bc6b9a4d73d4acee85d45078582642dd17e2c36e9f40a87631160c464d
Oct 31 00:45:40.635912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:40.635867    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc" event={"ID":"68a3cf8a-e155-4b83-9b09-4983a0bda04f","Type":"ContainerStarted","Data":"651a95bc6b9a4d73d4acee85d45078582642dd17e2c36e9f40a87631160c464d"}
Oct 31 00:45:41.399255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.399221    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"]
Oct 31 00:45:41.403645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.403605    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.411253 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.411226    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"]
Oct 31 00:45:41.471450 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.471416    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.471662 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.471644    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97pd\" (UniqueName: \"kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.471724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.471708    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.573238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.573205    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97pd\" (UniqueName: \"kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.573384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.573260    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.573384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.573296    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.573955 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.573935    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.575547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.575525    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.581476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.581454    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97pd\" (UniqueName: \"kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd\") pod \"maas-api-revoke-sub-78472ed3-hkgjt\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:41.645756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.645706    4924 generic.go:356] "Generic (PLEG): container finished" podID="68a3cf8a-e155-4b83-9b09-4983a0bda04f" containerID="fae45bc0b24b4ae8a6074d41ff5aa8503d8f588014273ea1584c0f30bb320ea6" exitCode=0
Oct 31 00:45:41.645939 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.645803    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc" event={"ID":"68a3cf8a-e155-4b83-9b09-4983a0bda04f","Type":"ContainerDied","Data":"fae45bc0b24b4ae8a6074d41ff5aa8503d8f588014273ea1584c0f30bb320ea6"}
Oct 31 00:45:41.721843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:41.721742    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:42.008379 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.008349    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"]
Oct 31 00:45:42.008554 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:42.008528    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9555a8e3_c99a_439c_a878_c213d78e3068.slice/crio-02498c313dda0f1ad53b989f1cd45d3735707dfa0abdfa5c83fcf86633e24336 WatchSource:0}: Error finding container 02498c313dda0f1ad53b989f1cd45d3735707dfa0abdfa5c83fcf86633e24336: Status 404 returned error can't find the container with id 02498c313dda0f1ad53b989f1cd45d3735707dfa0abdfa5c83fcf86633e24336
Oct 31 00:45:42.655642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.655554    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt" event={"ID":"9555a8e3-c99a-439c-a878-c213d78e3068","Type":"ContainerStarted","Data":"2914c451bd5be163c9ff2830e2e0447fd6cf883425daf9a8a678c223cade5c61"}
Oct 31 00:45:42.655642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.655610    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt" event={"ID":"9555a8e3-c99a-439c-a878-c213d78e3068","Type":"ContainerStarted","Data":"02498c313dda0f1ad53b989f1cd45d3735707dfa0abdfa5c83fcf86633e24336"}
Oct 31 00:45:42.675134 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.675086    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt" podStartSLOduration=1.675071799 podStartE2EDuration="1.675071799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:42.673864031 +0000 UTC m=+743.044647822" watchObservedRunningTime="2026-10-31 00:45:42.675071799 +0000 UTC m=+743.045855588"
Oct 31 00:45:42.684841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.684806    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"]
Oct 31 00:45:42.689603 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.689578    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.699146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.698431    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"]
Oct 31 00:45:42.789471 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.789439    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.789641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.789482    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmsc9\" (UniqueName: \"kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.789641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.789624    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.840330 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.840300    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:42.891002 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.890971    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.891197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.891169    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.891268 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.891215    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmsc9\" (UniqueName: \"kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.891729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.891709    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.893718 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.893687    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.921407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.921384    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmsc9\" (UniqueName: \"kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9\") pod \"maas-api-revoke-sub-8a503710-5jjzt\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:42.992355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.992324    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca\") pod \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") "
Oct 31 00:45:42.992355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.992373    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca\") pod \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") "
Oct 31 00:45:42.992604 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.992446    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d\" (UniqueName: \"kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d\") pod \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\" (UID: \"68a3cf8a-e155-4b83-9b09-4983a0bda04f\") "
Oct 31 00:45:42.992876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.992839    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca" pod "68a3cf8a-e155-4b83-9b09-4983a0bda04f" (UID: "68a3cf8a-e155-4b83-9b09-4983a0bda04f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:42.993027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.993007    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:42.994802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.994778    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d" pod "68a3cf8a-e155-4b83-9b09-4983a0bda04f" (UID: "68a3cf8a-e155-4b83-9b09-4983a0bda04f"). InnerVolumeSpecName "kube-api-access-pmp9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:42.994949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:42.994927    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca" pod "68a3cf8a-e155-4b83-9b09-4983a0bda04f" (UID: "68a3cf8a-e155-4b83-9b09-4983a0bda04f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:43.010585 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.010563    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:43.094941 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.094902    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmp9d\" (UniqueName: \"kubernetes.io/projected/68a3cf8a-e155-4b83-9b09-4983a0bda04f-kube-api-access-pmp9d\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:43.095094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.094943    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68a3cf8a-e155-4b83-9b09-4983a0bda04f-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:43.228273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.224918    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"]
Oct 31 00:45:43.228273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.226368    4924 cpu_manager.go:414] "containerMap: removing container" podUID="68a3cf8a-e155-4b83-9b09-4983a0bda04f" containerName="revoke-keys"
Oct 31 00:45:43.228273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.226387    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68a3cf8a-e155-4b83-9b09-4983a0bda04f" containerName="revoke-keys"
Oct 31 00:45:43.228273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.226709    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="68a3cf8a-e155-4b83-9b09-4983a0bda04f" containerName="revoke-keys"
Oct 31 00:45:43.233300 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.233152    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.240781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.240328    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"]
Oct 31 00:45:43.263841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.263809    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"]
Oct 31 00:45:43.268555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.268531    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.279477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.279417    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"]
Oct 31 00:45:43.318952 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.318919    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"]
Oct 31 00:45:43.320228 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:43.320202    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69083ca8_f0bb_4c02_88b7_e944b79b39a6.slice/crio-4de69028cbb2c1fa75b2118b420e73ced79aa98ea68de2171c2ab37b466dd070 WatchSource:0}: Error finding container 4de69028cbb2c1fa75b2118b420e73ced79aa98ea68de2171c2ab37b466dd070: Status 404 returned error can't find the container with id 4de69028cbb2c1fa75b2118b420e73ced79aa98ea68de2171c2ab37b466dd070
Oct 31 00:45:43.398120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398083    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4b4307-477d-44cd-a66a-66eb109867be-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398241    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398304 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398290    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjb7j\" (UniqueName: \"kubernetes.io/projected/9495ab3a-0b84-4854-bc5e-30d46c9f9040-kube-api-access-cjb7j\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.398372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398338    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.398484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398460    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398507    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398587    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.398703 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398686    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.398740 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398728    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmsm8\" (UniqueName: \"kubernetes.io/projected/fc4b4307-477d-44cd-a66a-66eb109867be-kube-api-access-vmsm8\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398813    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.398869 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398846    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4b4307-477d-44cd-a66a-66eb109867be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398873    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4b4307-477d-44cd-a66a-66eb109867be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.398985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398956    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.399026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.398997    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.399026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.399016    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.399135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.399077    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.399135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.399119    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.399205 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.399160    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500059 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.499984    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500059 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500044    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500065    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmsm8\" (UniqueName: \"kubernetes.io/projected/fc4b4307-477d-44cd-a66a-66eb109867be-kube-api-access-vmsm8\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500083    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500100    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4b4307-477d-44cd-a66a-66eb109867be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500117    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4b4307-477d-44cd-a66a-66eb109867be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500249    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500286    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500315    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500339    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500366    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500401    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500431    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4b4307-477d-44cd-a66a-66eb109867be-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500567    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500620    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjb7j\" (UniqueName: \"kubernetes.io/projected/9495ab3a-0b84-4854-bc5e-30d46c9f9040-kube-api-access-cjb7j\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500679    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500703    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.500875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500738    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.500875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500810    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.501189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.500885    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.501276 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.501234    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-workload-certs\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.501557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.501478    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.501557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.501494    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.501557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.501541    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc4b4307-477d-44cd-a66a-66eb109867be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.502256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.501774    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-workload-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.502256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.501925    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.502256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.502049    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-data\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.502256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.502082    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-credential-socket\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.503117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.503087    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.503117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.503110    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.503293 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.503252    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc4b4307-477d-44cd-a66a-66eb109867be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.503905 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.503869    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc4b4307-477d-44cd-a66a-66eb109867be-istio-envoy\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.509313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.509288    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjb7j\" (UniqueName: \"kubernetes.io/projected/9495ab3a-0b84-4854-bc5e-30d46c9f9040-kube-api-access-cjb7j\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.594530 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.594488    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmsm8\" (UniqueName: \"kubernetes.io/projected/fc4b4307-477d-44cd-a66a-66eb109867be-kube-api-access-vmsm8\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.594702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.594601    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9495ab3a-0b84-4854-bc5e-30d46c9f9040-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-9c7vk\" (UID: \"9495ab3a-0b84-4854-bc5e-30d46c9f9040\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.594802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.594783    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc4b4307-477d-44cd-a66a-66eb109867be-istio-token\") pod \"maas-default-gateway-openshift-default-6f54779779-pmzpl\" (UID: \"fc4b4307-477d-44cd-a66a-66eb109867be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:43.664819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.664786    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt" event={"ID":"69083ca8-f0bb-4c02-88b7-e944b79b39a6","Type":"ContainerStarted","Data":"4de69028cbb2c1fa75b2118b420e73ced79aa98ea68de2171c2ab37b466dd070"}
Oct 31 00:45:43.666758 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.666720    4924 generic.go:356] "Generic (PLEG): container finished" podID="9555a8e3-c99a-439c-a878-c213d78e3068" containerID="2914c451bd5be163c9ff2830e2e0447fd6cf883425daf9a8a678c223cade5c61" exitCode=0
Oct 31 00:45:43.666898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.666804    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt" event={"ID":"9555a8e3-c99a-439c-a878-c213d78e3068","Type":"ContainerDied","Data":"2914c451bd5be163c9ff2830e2e0447fd6cf883425daf9a8a678c223cade5c61"}
Oct 31 00:45:43.668245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.668219    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc" event={"ID":"68a3cf8a-e155-4b83-9b09-4983a0bda04f","Type":"ContainerDied","Data":"651a95bc6b9a4d73d4acee85d45078582642dd17e2c36e9f40a87631160c464d"}
Oct 31 00:45:43.668345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.668261    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="651a95bc6b9a4d73d4acee85d45078582642dd17e2c36e9f40a87631160c464d"
Oct 31 00:45:43.668345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.668229    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"
Oct 31 00:45:43.853549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.853513    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:43.891048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:43.891016    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:44.156424 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.156399    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"]
Oct 31 00:45:44.157714 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:44.157683    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9495ab3a_0b84_4854_bc5e_30d46c9f9040.slice/crio-5f20b3cc06086fa2bbdd32f0099c965d32b3863c488ec2759ab55ff278e07cf9 WatchSource:0}: Error finding container 5f20b3cc06086fa2bbdd32f0099c965d32b3863c488ec2759ab55ff278e07cf9: Status 404 returned error can't find the container with id 5f20b3cc06086fa2bbdd32f0099c965d32b3863c488ec2759ab55ff278e07cf9
Oct 31 00:45:44.160270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.160235    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:44.160365 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.160301    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:44.160365 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.160332    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:44.203653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.203624    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"]
Oct 31 00:45:44.209086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.209061    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.216274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.216251    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"]
Oct 31 00:45:44.219721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.219695    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"]
Oct 31 00:45:44.225569 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:44.223510    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4b4307_477d_44cd_a66a_66eb109867be.slice/crio-000f96a7907e0366a499e1c666e29fbdbcddaeb2b5a465910266973c55bf82a2 WatchSource:0}: Error finding container 000f96a7907e0366a499e1c666e29fbdbcddaeb2b5a465910266973c55bf82a2: Status 404 returned error can't find the container with id 000f96a7907e0366a499e1c666e29fbdbcddaeb2b5a465910266973c55bf82a2
Oct 31 00:45:44.229411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.229364    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:44.229519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.229447    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:44.229519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.229492    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:44.314694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.314657    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.314844 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.314767    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfgv\" (UniqueName: \"kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.314965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.314947    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.416113 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.415997    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.416274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.416177    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.416274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.416265    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfgv\" (UniqueName: \"kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.417021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.416984    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.418517 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.418492    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.424416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.424392    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfgv\" (UniqueName: \"kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv\") pod \"maas-api-revoke-sub-cd0199c3-fhlh9\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.552855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.552813    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:44.699948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.699560    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" event={"ID":"fc4b4307-477d-44cd-a66a-66eb109867be","Type":"ContainerStarted","Data":"a6324e79b14bf136870219cdb45c25794abe5fff079307754825cd711218aee5"}
Oct 31 00:45:44.699948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.699613    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" event={"ID":"fc4b4307-477d-44cd-a66a-66eb109867be","Type":"ContainerStarted","Data":"000f96a7907e0366a499e1c666e29fbdbcddaeb2b5a465910266973c55bf82a2"}
Oct 31 00:45:44.702939 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.702895    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" event={"ID":"9495ab3a-0b84-4854-bc5e-30d46c9f9040","Type":"ContainerStarted","Data":"4a7988361d1484bfea3dc6c93bdc3becf6253d054e6f42168667bbd767d93eef"}
Oct 31 00:45:44.702939 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.702938    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" event={"ID":"9495ab3a-0b84-4854-bc5e-30d46c9f9040","Type":"ContainerStarted","Data":"5f20b3cc06086fa2bbdd32f0099c965d32b3863c488ec2759ab55ff278e07cf9"}
Oct 31 00:45:44.708474 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.707845    4924 generic.go:356] "Generic (PLEG): container finished" podID="69083ca8-f0bb-4c02-88b7-e944b79b39a6" containerID="68fab8876983d02eaf5e6f2b91767d14a9dc936db709af7fec019add9021bd2c" exitCode=0
Oct 31 00:45:44.708474 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.708206    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt" event={"ID":"69083ca8-f0bb-4c02-88b7-e944b79b39a6","Type":"ContainerDied","Data":"68fab8876983d02eaf5e6f2b91767d14a9dc936db709af7fec019add9021bd2c"}
Oct 31 00:45:44.737518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.735805    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podStartSLOduration=1.735739766 podStartE2EDuration="1.735739766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:44.732110528 +0000 UTC m=+745.102894317" watchObservedRunningTime="2026-10-31 00:45:44.735739766 +0000 UTC m=+745.106523556"
Oct 31 00:45:44.771793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.769031    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podStartSLOduration=1.769007973 podStartE2EDuration="1.769007973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:44.761203688 +0000 UTC m=+745.131987494" watchObservedRunningTime="2026-10-31 00:45:44.769007973 +0000 UTC m=+745.139791763"
Oct 31 00:45:44.854695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.854531    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:44.855393 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.855364    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"]
Oct 31 00:45:44.860788 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:44.859280    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86bbcd08_30c2_42aa_a053_e55887c428a5.slice/crio-0fdf801c5f949c08a9a9e70599d7c18e19e20d1abf50e6ca66f58756d1cd1090 WatchSource:0}: Error finding container 0fdf801c5f949c08a9a9e70599d7c18e19e20d1abf50e6ca66f58756d1cd1090: Status 404 returned error can't find the container with id 0fdf801c5f949c08a9a9e70599d7c18e19e20d1abf50e6ca66f58756d1cd1090
Oct 31 00:45:44.901780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:44.891864    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:45.091854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.091819    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"]
Oct 31 00:45:45.171404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.171366    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.177241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.177209    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"]
Oct 31 00:45:45.177645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.177618    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7f1496\"/\"distinct-w1-7f1496-kserve-self-signed-certs\""
Oct 31 00:45:45.262013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.261933    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:45.272602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.272571    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.272801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.272651    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.272801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.272676    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.272801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.272691    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.272801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.272732    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6255\" (UniqueName: \"kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.272801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.272791    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.443803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.426201    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca\") pod \"9555a8e3-c99a-439c-a878-c213d78e3068\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") "
Oct 31 00:45:45.466061 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.466013    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd\" (UniqueName: \"kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd\") pod \"9555a8e3-c99a-439c-a878-c213d78e3068\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") "
Oct 31 00:45:45.466385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.466343    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca\") pod \"9555a8e3-c99a-439c-a878-c213d78e3068\" (UID: \"9555a8e3-c99a-439c-a878-c213d78e3068\") "
Oct 31 00:45:45.467426 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.467397    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.467721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.467701    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.468009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.467992    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.468160 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.468144    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.468279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.468264    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.468468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.468451    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6255\" (UniqueName: \"kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.469291 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.469260    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.469540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.469516    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.469811 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.469788    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.470109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.470085    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca" pod "9555a8e3-c99a-439c-a878-c213d78e3068" (UID: "9555a8e3-c99a-439c-a878-c213d78e3068"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:45.475152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.473793    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd" pod "9555a8e3-c99a-439c-a878-c213d78e3068" (UID: "9555a8e3-c99a-439c-a878-c213d78e3068"). InnerVolumeSpecName "kube-api-access-w97pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:45.475475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.475450    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca" pod "9555a8e3-c99a-439c-a878-c213d78e3068" (UID: "9555a8e3-c99a-439c-a878-c213d78e3068"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:45.475807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.475788    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.477312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.477291    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.477433 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.477408    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6255\" (UniqueName: \"kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255\") pod \"distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") " pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.569631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.569598    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w97pd\" (UniqueName: \"kubernetes.io/projected/9555a8e3-c99a-439c-a878-c213d78e3068-kube-api-access-w97pd\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:45.569631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.569627    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:45.569631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.569639    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9555a8e3-c99a-439c-a878-c213d78e3068-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:45.771990 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.771939    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:45.778527 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.778223    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"
Oct 31 00:45:45.786669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.786640    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt" event={"ID":"9555a8e3-c99a-439c-a878-c213d78e3068","Type":"ContainerDied","Data":"02498c313dda0f1ad53b989f1cd45d3735707dfa0abdfa5c83fcf86633e24336"}
Oct 31 00:45:45.786874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.786691    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02498c313dda0f1ad53b989f1cd45d3735707dfa0abdfa5c83fcf86633e24336"
Oct 31 00:45:45.786874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.786705    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9" event={"ID":"86bbcd08-30c2-42aa-a053-e55887c428a5","Type":"ContainerStarted","Data":"c38b7272e7c30523c9ab90d182d57869854f72c7fdfc12f32ddd123110f25a9c"}
Oct 31 00:45:45.786874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.786731    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9" event={"ID":"86bbcd08-30c2-42aa-a053-e55887c428a5","Type":"ContainerStarted","Data":"0fdf801c5f949c08a9a9e70599d7c18e19e20d1abf50e6ca66f58756d1cd1090"}
Oct 31 00:45:45.820825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.816368    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9" podStartSLOduration=1.816346273 podStartE2EDuration="1.816346273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:45.815178899 +0000 UTC m=+746.185962693" watchObservedRunningTime="2026-10-31 00:45:45.816346273 +0000 UTC m=+746.187130064"
Oct 31 00:45:45.862110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.860872    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:45.862110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.860933    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:45.905872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.905525    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:45.905872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:45.905589    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:46.263641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.263600    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"]
Oct 31 00:45:46.271876 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:46.270873    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf72a3bf_1d16_4cd3_a5a1_74ab07443025.slice/crio-15e25e620d8afa38d40511288c6027898b9c1296f17b15812b155bf3af4c9c85 WatchSource:0}: Error finding container 15e25e620d8afa38d40511288c6027898b9c1296f17b15812b155bf3af4c9c85: Status 404 returned error can't find the container with id 15e25e620d8afa38d40511288c6027898b9c1296f17b15812b155bf3af4c9c85
Oct 31 00:45:46.462595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.462434    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:46.583694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.583658    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca\") pod \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") "
Oct 31 00:45:46.583888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.583803    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9\" (UniqueName: \"kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9\") pod \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") "
Oct 31 00:45:46.583951 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.583884    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca\") pod \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\" (UID: \"69083ca8-f0bb-4c02-88b7-e944b79b39a6\") "
Oct 31 00:45:46.584532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.584507    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca" pod "69083ca8-f0bb-4c02-88b7-e944b79b39a6" (UID: "69083ca8-f0bb-4c02-88b7-e944b79b39a6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:46.585032 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.585014    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:46.587095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.587061    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9" pod "69083ca8-f0bb-4c02-88b7-e944b79b39a6" (UID: "69083ca8-f0bb-4c02-88b7-e944b79b39a6"). InnerVolumeSpecName "kube-api-access-pmsc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:46.587506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.587485    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca" pod "69083ca8-f0bb-4c02-88b7-e944b79b39a6" (UID: "69083ca8-f0bb-4c02-88b7-e944b79b39a6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:46.686896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.686740    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69083ca8-f0bb-4c02-88b7-e944b79b39a6-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:46.686896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.686846    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmsc9\" (UniqueName: \"kubernetes.io/projected/69083ca8-f0bb-4c02-88b7-e944b79b39a6-kube-api-access-pmsc9\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:46.789789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.789613    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"
Oct 31 00:45:46.789789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.789629    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt" event={"ID":"69083ca8-f0bb-4c02-88b7-e944b79b39a6","Type":"ContainerDied","Data":"4de69028cbb2c1fa75b2118b420e73ced79aa98ea68de2171c2ab37b466dd070"}
Oct 31 00:45:46.789789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.789687    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4de69028cbb2c1fa75b2118b420e73ced79aa98ea68de2171c2ab37b466dd070"
Oct 31 00:45:46.791268 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.791243    4924 generic.go:356] "Generic (PLEG): container finished" podID="86bbcd08-30c2-42aa-a053-e55887c428a5" containerID="c38b7272e7c30523c9ab90d182d57869854f72c7fdfc12f32ddd123110f25a9c" exitCode=0
Oct 31 00:45:46.791396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.791323    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9" event={"ID":"86bbcd08-30c2-42aa-a053-e55887c428a5","Type":"ContainerDied","Data":"c38b7272e7c30523c9ab90d182d57869854f72c7fdfc12f32ddd123110f25a9c"}
Oct 31 00:45:46.806769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.794153    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" event={"ID":"af72a3bf-1d16-4cd3-a5a1-74ab07443025","Type":"ContainerStarted","Data":"963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b"}
Oct 31 00:45:46.806769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.794187    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" event={"ID":"af72a3bf-1d16-4cd3-a5a1-74ab07443025","Type":"ContainerStarted","Data":"15e25e620d8afa38d40511288c6027898b9c1296f17b15812b155bf3af4c9c85"}
Oct 31 00:45:46.855388 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.855355    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:46.855582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.855555    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:46.869359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.869316    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podStartSLOduration=1.869304766 podStartE2EDuration="1.869304766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:46.867722468 +0000 UTC m=+747.238506259" watchObservedRunningTime="2026-10-31 00:45:46.869304766 +0000 UTC m=+747.240088554"
Oct 31 00:45:46.892069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.892043    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:46.892261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:46.892240    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:47.776216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:47.776181    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:47.853940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:47.853907    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:47.854565 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:47.854508    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:47.892230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:47.892179    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:47.892440 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:47.892420    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:48.184499 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.184472    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:48.361247 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.361148    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca\") pod \"86bbcd08-30c2-42aa-a053-e55887c428a5\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") "
Oct 31 00:45:48.361908 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.361879    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca\") pod \"86bbcd08-30c2-42aa-a053-e55887c428a5\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") "
Oct 31 00:45:48.362417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.362395    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv\" (UniqueName: \"kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv\") pod \"86bbcd08-30c2-42aa-a053-e55887c428a5\" (UID: \"86bbcd08-30c2-42aa-a053-e55887c428a5\") "
Oct 31 00:45:48.362802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.362421    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca" pod "86bbcd08-30c2-42aa-a053-e55887c428a5" (UID: "86bbcd08-30c2-42aa-a053-e55887c428a5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:48.363453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.363430    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:48.367039 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.366992    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv" pod "86bbcd08-30c2-42aa-a053-e55887c428a5" (UID: "86bbcd08-30c2-42aa-a053-e55887c428a5"). InnerVolumeSpecName "kube-api-access-qqfgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:48.367296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.367269    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca" pod "86bbcd08-30c2-42aa-a053-e55887c428a5" (UID: "86bbcd08-30c2-42aa-a053-e55887c428a5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:48.464638 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.464550    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86bbcd08-30c2-42aa-a053-e55887c428a5-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:48.464638 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.464583    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqfgv\" (UniqueName: \"kubernetes.io/projected/86bbcd08-30c2-42aa-a053-e55887c428a5-kube-api-access-qqfgv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:48.859644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.859608    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:48.860208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.860156    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:48.865189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.865163    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"
Oct 31 00:45:48.865353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.865239    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9" event={"ID":"86bbcd08-30c2-42aa-a053-e55887c428a5","Type":"ContainerDied","Data":"0fdf801c5f949c08a9a9e70599d7c18e19e20d1abf50e6ca66f58756d1cd1090"}
Oct 31 00:45:48.865353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.865284    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fdf801c5f949c08a9a9e70599d7c18e19e20d1abf50e6ca66f58756d1cd1090"
Oct 31 00:45:48.892217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.892175    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:48.892355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:48.892232    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:49.855239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:49.855192    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:49.855435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:49.855259    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:49.892101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:49.892064    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:49.892601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:49.892138    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:50.854677 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:50.854633    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:50.854909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:50.854704    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:50.892083 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:50.892042    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:50.892259 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:50.892115    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:51.858770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:51.857191    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 31 00:45:51.858770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:51.857255    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 31 00:45:51.902770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:51.902562    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:51.902770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:51.902619    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:52.752790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.751899    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:45:52.880190 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.880150    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"]
Oct 31 00:45:52.881141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881106    4924 cpu_manager.go:414] "containerMap: removing container" podUID="69083ca8-f0bb-4c02-88b7-e944b79b39a6" containerName="revoke-keys"
Oct 31 00:45:52.881141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881136    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69083ca8-f0bb-4c02-88b7-e944b79b39a6" containerName="revoke-keys"
Oct 31 00:45:52.881373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881180    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9555a8e3-c99a-439c-a878-c213d78e3068" containerName="revoke-keys"
Oct 31 00:45:52.881373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881189    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9555a8e3-c99a-439c-a878-c213d78e3068" containerName="revoke-keys"
Oct 31 00:45:52.881373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881200    4924 cpu_manager.go:414] "containerMap: removing container" podUID="86bbcd08-30c2-42aa-a053-e55887c428a5" containerName="revoke-keys"
Oct 31 00:45:52.881373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881208    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86bbcd08-30c2-42aa-a053-e55887c428a5" containerName="revoke-keys"
Oct 31 00:45:52.881373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881343    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9555a8e3-c99a-439c-a878-c213d78e3068" containerName="revoke-keys"
Oct 31 00:45:52.881373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881360    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="86bbcd08-30c2-42aa-a053-e55887c428a5" containerName="revoke-keys"
Oct 31 00:45:52.881675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.881382    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="69083ca8-f0bb-4c02-88b7-e944b79b39a6" containerName="revoke-keys"
Oct 31 00:45:52.892695 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.892665    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Oct 31 00:45:52.892875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.892722    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Oct 31 00:45:52.913766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.912953    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"]
Oct 31 00:45:52.913766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.913097    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:52.957050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.953265    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:45:52.957050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:52.953344    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:45:53.061958 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.061911    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.062138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.062116    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmrr7\" (UniqueName: \"kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.062224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.062161    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.163528 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.163492    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmrr7\" (UniqueName: \"kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.163707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.163546    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.163707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.163652    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.164516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.164490    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.166809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.166786    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.172714 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.172688    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmrr7\" (UniqueName: \"kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7\") pod \"maas-api-revoke-sub-aa9f6c91-svxmq\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.387366 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.387250    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:53.748635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.748590    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"]
Oct 31 00:45:53.750089 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:53.750061    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86404c9f_c131_436a_94c9_8da6c9a752be.slice/crio-9e24d8b927c9b89a5c157ca5b86b11204c88f72e42b4b22f3bd20b5a1dd1efa4 WatchSource:0}: Error finding container 9e24d8b927c9b89a5c157ca5b86b11204c88f72e42b4b22f3bd20b5a1dd1efa4: Status 404 returned error can't find the container with id 9e24d8b927c9b89a5c157ca5b86b11204c88f72e42b4b22f3bd20b5a1dd1efa4
Oct 31 00:45:53.914932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.914886    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:45:53.915452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:53.915420    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:45:54.087769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.084441    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq" event={"ID":"86404c9f-c131-436a-94c9-8da6c9a752be","Type":"ContainerStarted","Data":"9e24d8b927c9b89a5c157ca5b86b11204c88f72e42b4b22f3bd20b5a1dd1efa4"}
Oct 31 00:45:54.087769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.087390    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:54.089872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.088848    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:54.095058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.095001    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk"
Oct 31 00:45:54.250316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.250278    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:54.253238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.252647    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:54.253238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.252729    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl"
Oct 31 00:45:54.499243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.499163    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"]
Oct 31 00:45:54.503403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.503383    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.510491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.510461    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"]
Oct 31 00:45:54.588296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.588259    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.588475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.588397    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.588475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.588447    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8l4b\" (UniqueName: \"kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.689558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.689525    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.689809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.689608    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.689809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.689665    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8l4b\" (UniqueName: \"kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.690378 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.690345    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.693017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.692989    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.699337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.699311    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8l4b\" (UniqueName: \"kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b\") pod \"maas-api-revoke-sub-d517c3c3-ph785\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:54.824404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:54.824378    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:55.096473 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:55.096380    4924 generic.go:356] "Generic (PLEG): container finished" podID="86404c9f-c131-436a-94c9-8da6c9a752be" containerID="0ba8e497071218ae2b3d09d2b6b0c053c66a8117e28568935a4567e240db4b8f" exitCode=0
Oct 31 00:45:55.096913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:55.096462    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq" event={"ID":"86404c9f-c131-436a-94c9-8da6c9a752be","Type":"ContainerDied","Data":"0ba8e497071218ae2b3d09d2b6b0c053c66a8117e28568935a4567e240db4b8f"}
Oct 31 00:45:55.112135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:55.112108    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"]
Oct 31 00:45:55.112941 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:55.112917    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da422dc_6ecc_4c70_a01f_1ac212ee9944.slice/crio-0c0abd5673f7c767bced3384d7a640795776ad873ef76c6552fb414c0e25cb27 WatchSource:0}: Error finding container 0c0abd5673f7c767bced3384d7a640795776ad873ef76c6552fb414c0e25cb27: Status 404 returned error can't find the container with id 0c0abd5673f7c767bced3384d7a640795776ad873ef76c6552fb414c0e25cb27
Oct 31 00:45:55.776048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:55.776020    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:55.789286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:55.789261    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:56.104858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.104762    4924 generic.go:356] "Generic (PLEG): container finished" podID="1da422dc-6ecc-4c70-a01f-1ac212ee9944" containerID="f7fff139103c70d22d0884bf652ee78b35c3df4d87ecc552fcfa7d98398e83b8" exitCode=0
Oct 31 00:45:56.104858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.104845    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785" event={"ID":"1da422dc-6ecc-4c70-a01f-1ac212ee9944","Type":"ContainerDied","Data":"f7fff139103c70d22d0884bf652ee78b35c3df4d87ecc552fcfa7d98398e83b8"}
Oct 31 00:45:56.105363 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.104899    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785" event={"ID":"1da422dc-6ecc-4c70-a01f-1ac212ee9944","Type":"ContainerStarted","Data":"0c0abd5673f7c767bced3384d7a640795776ad873ef76c6552fb414c0e25cb27"}
Oct 31 00:45:56.117445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.117416    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:45:56.315907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.315885    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:56.413505 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.413411    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7\" (UniqueName: \"kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7\") pod \"86404c9f-c131-436a-94c9-8da6c9a752be\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") "
Oct 31 00:45:56.413676 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.413501    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca\") pod \"86404c9f-c131-436a-94c9-8da6c9a752be\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") "
Oct 31 00:45:56.413759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.413710    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca\") pod \"86404c9f-c131-436a-94c9-8da6c9a752be\" (UID: \"86404c9f-c131-436a-94c9-8da6c9a752be\") "
Oct 31 00:45:56.415319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.415064    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca" pod "86404c9f-c131-436a-94c9-8da6c9a752be" (UID: "86404c9f-c131-436a-94c9-8da6c9a752be"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:56.427313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.427270    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca" pod "86404c9f-c131-436a-94c9-8da6c9a752be" (UID: "86404c9f-c131-436a-94c9-8da6c9a752be"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:56.430544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.430513    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7" pod "86404c9f-c131-436a-94c9-8da6c9a752be" (UID: "86404c9f-c131-436a-94c9-8da6c9a752be"). InnerVolumeSpecName "kube-api-access-jmrr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:56.515357 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.515317    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmrr7\" (UniqueName: \"kubernetes.io/projected/86404c9f-c131-436a-94c9-8da6c9a752be-kube-api-access-jmrr7\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:56.515357 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.515349    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:56.515357 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.515361    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86404c9f-c131-436a-94c9-8da6c9a752be-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:56.917219 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.917063    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"]
Oct 31 00:45:56.919836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.919248    4924 cpu_manager.go:414] "containerMap: removing container" podUID="86404c9f-c131-436a-94c9-8da6c9a752be" containerName="revoke-keys"
Oct 31 00:45:56.919836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.919270    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86404c9f-c131-436a-94c9-8da6c9a752be" containerName="revoke-keys"
Oct 31 00:45:56.919836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.919481    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="86404c9f-c131-436a-94c9-8da6c9a752be" containerName="revoke-keys"
Oct 31 00:45:56.923800 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.923775    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:56.924177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.924151    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"]
Oct 31 00:45:56.928233 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.928202    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-47f370\"/\"kube-root-ca.crt\""
Oct 31 00:45:56.928500 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.928483    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-47f370\"/\"distinct-2-w6-47f370-kserve-self-signed-certs\""
Oct 31 00:45:56.928576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.928530    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-47f370\"/\"openshift-service-ca.crt\""
Oct 31 00:45:56.928627 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:56.928595    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-47f370\"/\"default-dockercfg-42hms\""
Oct 31 00:45:57.023673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.023638    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.023673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.023673    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptpch\" (UniqueName: \"kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.023932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.023862    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.023932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.023880    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.023932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.023907    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.024077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.023956    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.118765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.118170    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq" event={"ID":"86404c9f-c131-436a-94c9-8da6c9a752be","Type":"ContainerDied","Data":"9e24d8b927c9b89a5c157ca5b86b11204c88f72e42b4b22f3bd20b5a1dd1efa4"}
Oct 31 00:45:57.118765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.118233    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e24d8b927c9b89a5c157ca5b86b11204c88f72e42b4b22f3bd20b5a1dd1efa4"
Oct 31 00:45:57.118765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.118314    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"
Oct 31 00:45:57.129189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.128893    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.129189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.128939    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.129189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.128982    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.129189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.129022    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.129189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.129082    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.129189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.129144    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptpch\" (UniqueName: \"kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.131372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.129765    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.131372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.130036    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.131372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.130610    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.134304 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.134280    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.141429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.141403    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.144235 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.144207    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptpch\" (UniqueName: \"kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch\") pod \"distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") " pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.243870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.243719    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:57.274384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.274355    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:57.435828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.435789    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca\") pod \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") "
Oct 31 00:45:57.436006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.435951    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b\" (UniqueName: \"kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b\") pod \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") "
Oct 31 00:45:57.436064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.436028    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca\") pod \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\" (UID: \"1da422dc-6ecc-4c70-a01f-1ac212ee9944\") "
Oct 31 00:45:57.436398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.436365    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca" pod "1da422dc-6ecc-4c70-a01f-1ac212ee9944" (UID: "1da422dc-6ecc-4c70-a01f-1ac212ee9944"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:57.436540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.436474    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:57.438177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.438149    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca" pod "1da422dc-6ecc-4c70-a01f-1ac212ee9944" (UID: "1da422dc-6ecc-4c70-a01f-1ac212ee9944"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:45:57.438344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.438322    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b" pod "1da422dc-6ecc-4c70-a01f-1ac212ee9944" (UID: "1da422dc-6ecc-4c70-a01f-1ac212ee9944"). InnerVolumeSpecName "kube-api-access-b8l4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:57.538068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.538019    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1da422dc-6ecc-4c70-a01f-1ac212ee9944-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:57.538068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.538061    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8l4b\" (UniqueName: \"kubernetes.io/projected/1da422dc-6ecc-4c70-a01f-1ac212ee9944-kube-api-access-b8l4b\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:57.543594 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:57.543565    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55d7c9b5_8371_493f_a948_842d079ab361.slice/crio-73e1ec55a5706d50c96eb8c6b31d2d88644110e94e13b148c8b64aaea88408b3 WatchSource:0}: Error finding container 73e1ec55a5706d50c96eb8c6b31d2d88644110e94e13b148c8b64aaea88408b3: Status 404 returned error can't find the container with id 73e1ec55a5706d50c96eb8c6b31d2d88644110e94e13b148c8b64aaea88408b3
Oct 31 00:45:57.543726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.543659    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"]
Oct 31 00:45:57.576978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.576943    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"]
Oct 31 00:45:57.577529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.577514    4924 cpu_manager.go:414] "containerMap: removing container" podUID="1da422dc-6ecc-4c70-a01f-1ac212ee9944" containerName="revoke-keys"
Oct 31 00:45:57.577529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.577528    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1da422dc-6ecc-4c70-a01f-1ac212ee9944" containerName="revoke-keys"
Oct 31 00:45:57.577665 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.577605    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="1da422dc-6ecc-4c70-a01f-1ac212ee9944" containerName="revoke-keys"
Oct 31 00:45:57.582200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.582178    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.593497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.593470    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"]
Oct 31 00:45:57.741007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.740908    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.740966    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741160    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741208    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741376 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741345    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5lfn\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741404    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741469    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741535    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.741618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.741577    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842397    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5lfn\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842446    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842505    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842576    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842699    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842796    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.842848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842831    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.842979    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.843023    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.843283    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.843459    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.843515    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.843581    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.843999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.843972    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.846017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.845997    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.846155 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.846131    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.854718 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:45:57.854695    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:45:57.859065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.859045    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5lfn\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.859589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.859572    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token\") pod \"e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") " pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.919634 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.919598    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:45:57.966928 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.966866    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"]
Oct 31 00:45:57.972535 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:57.972506    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.012915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.012838    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"]
Oct 31 00:45:58.109373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.109337    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"]
Oct 31 00:45:58.148012 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.144716    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.157696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.157653    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.157885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.157809    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.157885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.157848    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.158006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.157916    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.158006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.157963    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.158113 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.158055    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.158176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.158122    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.158176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.158146    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.158275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.158200    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sbnd\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.159221 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.159182    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"]
Oct 31 00:45:58.164471 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.164441    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" event={"ID":"55d7c9b5-8371-493f-a948-842d079ab361","Type":"ContainerStarted","Data":"ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84"}
Oct 31 00:45:58.164552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.164488    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" event={"ID":"55d7c9b5-8371-493f-a948-842d079ab361","Type":"ContainerStarted","Data":"73e1ec55a5706d50c96eb8c6b31d2d88644110e94e13b148c8b64aaea88408b3"}
Oct 31 00:45:58.167333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.167313    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785" event={"ID":"1da422dc-6ecc-4c70-a01f-1ac212ee9944","Type":"ContainerDied","Data":"0c0abd5673f7c767bced3384d7a640795776ad873ef76c6552fb414c0e25cb27"}
Oct 31 00:45:58.167491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.167474    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c0abd5673f7c767bced3384d7a640795776ad873ef76c6552fb414c0e25cb27"
Oct 31 00:45:58.167491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.167416    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"
Oct 31 00:45:58.194217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.194161    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podStartSLOduration=2.194143705 podStartE2EDuration="2.194143705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:58.18340557 +0000 UTC m=+758.554189357" watchObservedRunningTime="2026-10-31 00:45:58.194143705 +0000 UTC m=+758.564927495"
Oct 31 00:45:58.214086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.214058    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"]
Oct 31 00:45:58.215671 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:58.215644    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6c6f7d_da40_4c1f_8b0d_44c02cfe4d30.slice/crio-954e87b26071f601136ed8e987c68b337bf81e7ebd6e8629d586b52a6e59756b WatchSource:0}: Error finding container 954e87b26071f601136ed8e987c68b337bf81e7ebd6e8629d586b52a6e59756b: Status 404 returned error can't find the container with id 954e87b26071f601136ed8e987c68b337bf81e7ebd6e8629d586b52a6e59756b
Oct 31 00:45:58.218619 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.218584    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.218719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.218660    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.218719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.218702    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.244649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.244626    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:45:58.259121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259093    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259169    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259202    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.259332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259246    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259270    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.259449 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259435    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.259497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259469    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.259529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259506    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259558    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259605    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.259653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259640    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.259725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259709    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259783    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.259916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259895    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.260000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259981    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbnd\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.260054 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.259999    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.260054 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260023    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.260156 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260083    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.260156 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260088    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.260259 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260154    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.260259 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260195    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67s4f\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.260259 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260254    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.260405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.260305    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.262039 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.262017    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.262364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.262344    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.268032 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.268007    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sbnd\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.268300 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.268282    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.293267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.293245    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361289    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361340    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361371    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361391    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361429    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361449    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361500    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361558    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.361581    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67s4f\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.362211    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.362980    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.363044    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.363167    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.363409    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.366791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.365000    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.369128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.369080    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.369266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.369248    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67s4f\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.370142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.370117    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.478794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.478759    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:58.636728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.636689    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"]
Oct 31 00:45:58.646203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.646156    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.646419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.646389    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.646520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.646447    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.779631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.779562    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"]
Oct 31 00:45:58.783966 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:45:58.783937    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb74ba3_f886_4ee9_80e6_d85a149092a7.slice/crio-b61b6e171439c2bdc25010834f565a02304c7598d1be7b86b7089a4996f9893b WatchSource:0}: Error finding container b61b6e171439c2bdc25010834f565a02304c7598d1be7b86b7089a4996f9893b: Status 404 returned error can't find the container with id b61b6e171439c2bdc25010834f565a02304c7598d1be7b86b7089a4996f9893b
Oct 31 00:45:58.789820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.788956    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.789820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.789068    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:58.789820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:58.789128    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:45:59.180625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.180461    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" event={"ID":"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30","Type":"ContainerStarted","Data":"6863e752021b11433718bd0183023e98982d57b28fb8db0f7446e8c571554cfd"}
Oct 31 00:45:59.180625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.180527    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" event={"ID":"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30","Type":"ContainerStarted","Data":"954e87b26071f601136ed8e987c68b337bf81e7ebd6e8629d586b52a6e59756b"}
Oct 31 00:45:59.182352 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.182325    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" event={"ID":"5ae76819-9bd6-4f27-9fa9-b63ff98440fb","Type":"ContainerStarted","Data":"7e1ede373e466bfebc9a54cf6b3b0481f898601b3ceb08d45e7ddad37a3e533e"}
Oct 31 00:45:59.182460 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.182365    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" event={"ID":"5ae76819-9bd6-4f27-9fa9-b63ff98440fb","Type":"ContainerStarted","Data":"f7ab4f11be0194cb2b84de7adfa5b0f4a83f12849ce51835fe708cb3471863a4"}
Oct 31 00:45:59.184651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.184600    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" event={"ID":"8cb74ba3-f886-4ee9-80e6-d85a149092a7","Type":"ContainerStarted","Data":"1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51"}
Oct 31 00:45:59.184803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.184654    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" event={"ID":"8cb74ba3-f886-4ee9-80e6-d85a149092a7","Type":"ContainerStarted","Data":"b61b6e171439c2bdc25010834f565a02304c7598d1be7b86b7089a4996f9893b"}
Oct 31 00:45:59.237770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.237382    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podStartSLOduration=2.237364452 podStartE2EDuration="2.237364452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:59.234154279 +0000 UTC m=+759.604938069" watchObservedRunningTime="2026-10-31 00:45:59.237364452 +0000 UTC m=+759.608148243"
Oct 31 00:45:59.255998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.255308    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podStartSLOduration=1.25529928 podStartE2EDuration="1.25529928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:59.252645555 +0000 UTC m=+759.623429343" watchObservedRunningTime="2026-10-31 00:45:59.25529928 +0000 UTC m=+759.626083131"
Oct 31 00:45:59.271335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.271279    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podStartSLOduration=2.27126296 podStartE2EDuration="2.27126296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:59.269614499 +0000 UTC m=+759.640398287" watchObservedRunningTime="2026-10-31 00:45:59.27126296 +0000 UTC m=+759.642046818"
Oct 31 00:45:59.310770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.298152    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:45:59.310770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.306583    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:45:59.310770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.306645    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:45:59.479646 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.479535    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:45:59.480787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.480762    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:45:59.480886 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.480821    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:45:59.964770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.963390    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:45:59.964770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:45:59.963453    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 31 00:46:00.239825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.209587    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:46:00.298260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.298214    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:46:00.298529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.298504    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:46:00.479584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.479550    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:46:00.480162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.480129    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:46:00.922771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.921804    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:46:00.922771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:00.921866    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 31 00:46:01.297197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:01.297156    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:46:01.297850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:01.297817    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:46:01.479239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:01.479199    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:46:01.479526 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:01.479494    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:46:01.933768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:01.933558    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:46:01.933768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:01.933639    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 31 00:46:02.337928 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:02.337885    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:46:02.338559 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:02.338525    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:46:02.494796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:02.494454    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:46:02.494796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:02.494530    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:46:02.923780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:02.922441    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:46:02.923780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:02.922518    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 31 00:46:03.337771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:03.337137    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:46:03.337771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:03.337229    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:46:03.523769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:03.523136    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:46:03.523769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:03.523200    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:46:03.929937 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:03.929902    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:46:03.930256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:03.930214    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 31 00:46:04.310773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:04.309888    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:46:04.310773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:04.309956    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:46:04.480932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:04.479144    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:46:04.480932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:04.479215    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:46:04.921215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:04.921173    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:46:04.921518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:04.921482    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 31 00:46:05.307772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.297046    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 31 00:46:05.307772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.297129    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 31 00:46:05.479976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.479928    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 31 00:46:05.480181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.479997    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 31 00:46:05.933102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.933064    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:46:05.934292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.934258    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:46:05.952944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:05.952910    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:46:06.299569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:06.299527    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:46:06.300453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:06.300428    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:46:06.301195 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:06.301173    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:46:06.534824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:06.534793    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:46:06.535809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:06.535786    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:46:06.537183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:06.537161    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:46:07.244821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:07.244787    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:07.260757 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:07.260705    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:07.735242 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:07.735212    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:09.295813 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.295680    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"]
Oct 31 00:46:09.314022 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.313988    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"]
Oct 31 00:46:09.314406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.314339    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.328225 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.328196    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:46:09.341915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.338299    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:46:09.483098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.482579    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.483098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.482683    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h6ls\" (UniqueName: \"kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.483098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.482954    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.517605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.517174    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"]
Oct 31 00:46:09.527109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.527083    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.533475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.533451    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"]
Oct 31 00:46:09.544764 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.541363    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d4a7e\"/\"distinct-w3-1d4a7e-kserve-self-signed-certs\""
Oct 31 00:46:09.584766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.584663    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.585018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.584996    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.585604 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.585569    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h6ls\" (UniqueName: \"kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.587267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.587225    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.589918 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.589847    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.631766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.631314    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h6ls\" (UniqueName: \"kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls\") pod \"maas-api-revoke-sub-c01141f6-s6jzz\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.679581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.679545    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:09.687651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.687619    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.687857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.687831    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.688047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.688029    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.688186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.688170    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.688318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.688302    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.688512 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.688494    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx2nh\" (UniqueName: \"kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.806501    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.806563    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.806663    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx2nh\" (UniqueName: \"kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.806828    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.806857    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.806944    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.807969    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.808190    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.809774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.808409    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.836829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.824721    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.859793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.841292    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx2nh\" (UniqueName: \"kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.859793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.841974    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm\") pod \"distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") " pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:09.895786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:09.884835    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:11.178277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.128594    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:11.220766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.149446    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"]
Oct 31 00:46:11.457130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.444868    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:11.473062 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.471975    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:11.749297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.749202    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:11.764381 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.751051    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:46:11.929980 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.929934    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:11.977775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:11.977689    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:46:12.132364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:12.130852    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:12.382306 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:12.382160    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz" event={"ID":"cf9eff68-370b-44ef-8d0f-6cc1715c79f8","Type":"ContainerStarted","Data":"953b13a98979af44e63c66155987df6d202c56037d9702372c3b36c4da3afa7c"}
Oct 31 00:46:12.610920 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:12.610883    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"]
Oct 31 00:46:12.619985 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:46:12.619940    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1df650ec_c09d_4e4d_8ebb_6de4cc5ac6c6.slice/crio-3834ffcd48be88f6b4510e5b07e992ba6bac4452884fc03c78b4b4556b4712a0 WatchSource:0}: Error finding container 3834ffcd48be88f6b4510e5b07e992ba6bac4452884fc03c78b4b4556b4712a0: Status 404 returned error can't find the container with id 3834ffcd48be88f6b4510e5b07e992ba6bac4452884fc03c78b4b4556b4712a0
Oct 31 00:46:13.395672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:13.395638    4924 generic.go:356] "Generic (PLEG): container finished" podID="cf9eff68-370b-44ef-8d0f-6cc1715c79f8" containerID="e76fe878d7b35519391b31cd9b4f70fa0fad93972ce9a0a2d8438e05e6e2fadc" exitCode=0
Oct 31 00:46:13.396129 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:13.395720    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz" event={"ID":"cf9eff68-370b-44ef-8d0f-6cc1715c79f8","Type":"ContainerDied","Data":"e76fe878d7b35519391b31cd9b4f70fa0fad93972ce9a0a2d8438e05e6e2fadc"}
Oct 31 00:46:13.397729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:13.397681    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" event={"ID":"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6","Type":"ContainerStarted","Data":"8b5a1c9ad3f33646336db16a4be7b4f5e77277bd116f64488ac4ce733aea8a42"}
Oct 31 00:46:13.397877 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:13.397736    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" event={"ID":"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6","Type":"ContainerStarted","Data":"3834ffcd48be88f6b4510e5b07e992ba6bac4452884fc03c78b4b4556b4712a0"}
Oct 31 00:46:13.437277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:13.437209    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podStartSLOduration=4.4371882 podStartE2EDuration="4.4371882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:13.430075534 +0000 UTC m=+773.800859324" watchObservedRunningTime="2026-10-31 00:46:13.4371882 +0000 UTC m=+773.807971989"
Oct 31 00:46:13.886253 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:13.886213    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:14.588215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.588187    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:14.653529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.653487    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca\") pod \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") "
Oct 31 00:46:14.653728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.653629    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca\") pod \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") "
Oct 31 00:46:14.653728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.653713    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls\" (UniqueName: \"kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls\") pod \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\" (UID: \"cf9eff68-370b-44ef-8d0f-6cc1715c79f8\") "
Oct 31 00:46:14.654037 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.654010    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca" pod "cf9eff68-370b-44ef-8d0f-6cc1715c79f8" (UID: "cf9eff68-370b-44ef-8d0f-6cc1715c79f8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:46:14.654404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.654382    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:14.655721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.655697    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca" pod "cf9eff68-370b-44ef-8d0f-6cc1715c79f8" (UID: "cf9eff68-370b-44ef-8d0f-6cc1715c79f8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:46:14.655891 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.655795    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls" pod "cf9eff68-370b-44ef-8d0f-6cc1715c79f8" (UID: "cf9eff68-370b-44ef-8d0f-6cc1715c79f8"). InnerVolumeSpecName "kube-api-access-7h6ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:46:14.755474 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.755438    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:14.755474 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:14.755471    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h6ls\" (UniqueName: \"kubernetes.io/projected/cf9eff68-370b-44ef-8d0f-6cc1715c79f8-kube-api-access-7h6ls\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:15.415490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:15.415456    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"
Oct 31 00:46:15.415490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:15.415471    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz" event={"ID":"cf9eff68-370b-44ef-8d0f-6cc1715c79f8","Type":"ContainerDied","Data":"953b13a98979af44e63c66155987df6d202c56037d9702372c3b36c4da3afa7c"}
Oct 31 00:46:15.415777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:15.415522    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="953b13a98979af44e63c66155987df6d202c56037d9702372c3b36c4da3afa7c"
Oct 31 00:46:18.117788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.115579    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"]
Oct 31 00:46:18.117788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.117065    4924 cpu_manager.go:414] "containerMap: removing container" podUID="cf9eff68-370b-44ef-8d0f-6cc1715c79f8" containerName="revoke-keys"
Oct 31 00:46:18.117788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.117086    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf9eff68-370b-44ef-8d0f-6cc1715c79f8" containerName="revoke-keys"
Oct 31 00:46:18.117788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.117393    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf9eff68-370b-44ef-8d0f-6cc1715c79f8" containerName="revoke-keys"
Oct 31 00:46:18.128340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.128306    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.133086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.133055    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:46:18.133245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.133103    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:46:18.134876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.134849    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"]
Oct 31 00:46:18.200372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.200337    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s6f7\" (UniqueName: \"kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.200569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.200397    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.200569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.200542    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.302256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.302214    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s6f7\" (UniqueName: \"kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.302455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.302290    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.302455 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.302357    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.303024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.302989    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.304730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.304704    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.310629 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.310602    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s6f7\" (UniqueName: \"kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7\") pod \"maas-api-revoke-sub-bea488a5-rppfq\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.455681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.455595    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:18.751066 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:18.751036    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"]
Oct 31 00:46:18.751352 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:46:18.751325    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf0b95f_fbed_4e37_b599_e164e93c17c9.slice/crio-73a837c8422d9af6cc860dea615ef8bd1157f325261ec1bbd05f125f031f28a2 WatchSource:0}: Error finding container 73a837c8422d9af6cc860dea615ef8bd1157f325261ec1bbd05f125f031f28a2: Status 404 returned error can't find the container with id 73a837c8422d9af6cc860dea615ef8bd1157f325261ec1bbd05f125f031f28a2
Oct 31 00:46:19.448147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:19.448111    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq" event={"ID":"6cf0b95f-fbed-4e37-b599-e164e93c17c9","Type":"ContainerStarted","Data":"c29441fd7d455eb8ab38793a0440c7b430e934c08a39ff2aabdfdd5fe1f58b05"}
Oct 31 00:46:19.448596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:19.448166    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq" event={"ID":"6cf0b95f-fbed-4e37-b599-e164e93c17c9","Type":"ContainerStarted","Data":"73a837c8422d9af6cc860dea615ef8bd1157f325261ec1bbd05f125f031f28a2"}
Oct 31 00:46:19.476270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:19.476220    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq" podStartSLOduration=1.476207008 podStartE2EDuration="1.476207008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:19.475613257 +0000 UTC m=+779.846397055" watchObservedRunningTime="2026-10-31 00:46:19.476207008 +0000 UTC m=+779.846990794"
Oct 31 00:46:19.885972 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:19.885936    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:19.899820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:19.899795    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:20.457976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:20.457944    4924 generic.go:356] "Generic (PLEG): container finished" podID="6cf0b95f-fbed-4e37-b599-e164e93c17c9" containerID="c29441fd7d455eb8ab38793a0440c7b430e934c08a39ff2aabdfdd5fe1f58b05" exitCode=0
Oct 31 00:46:20.458392 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:20.458027    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq" event={"ID":"6cf0b95f-fbed-4e37-b599-e164e93c17c9","Type":"ContainerDied","Data":"c29441fd7d455eb8ab38793a0440c7b430e934c08a39ff2aabdfdd5fe1f58b05"}
Oct 31 00:46:20.469248 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:20.469222    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:46:21.645562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.645534    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:21.754714 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.754619    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca\") pod \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") "
Oct 31 00:46:21.754714 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.754675    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7\" (UniqueName: \"kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7\") pod \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") "
Oct 31 00:46:21.754986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.754720    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca\") pod \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\" (UID: \"6cf0b95f-fbed-4e37-b599-e164e93c17c9\") "
Oct 31 00:46:21.755076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.755048    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca" pod "6cf0b95f-fbed-4e37-b599-e164e93c17c9" (UID: "6cf0b95f-fbed-4e37-b599-e164e93c17c9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:46:21.755390 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.755369    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:21.756856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.756823    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca" pod "6cf0b95f-fbed-4e37-b599-e164e93c17c9" (UID: "6cf0b95f-fbed-4e37-b599-e164e93c17c9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:46:21.757129 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.757111    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7" pod "6cf0b95f-fbed-4e37-b599-e164e93c17c9" (UID: "6cf0b95f-fbed-4e37-b599-e164e93c17c9"). InnerVolumeSpecName "kube-api-access-8s6f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:46:21.857091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.857055    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s6f7\" (UniqueName: \"kubernetes.io/projected/6cf0b95f-fbed-4e37-b599-e164e93c17c9-kube-api-access-8s6f7\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:21.857091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:21.857090    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6cf0b95f-fbed-4e37-b599-e164e93c17c9-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:22.474786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:22.474759    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"
Oct 31 00:46:22.474786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:22.474771    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq" event={"ID":"6cf0b95f-fbed-4e37-b599-e164e93c17c9","Type":"ContainerDied","Data":"73a837c8422d9af6cc860dea615ef8bd1157f325261ec1bbd05f125f031f28a2"}
Oct 31 00:46:22.475027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:22.474818    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73a837c8422d9af6cc860dea615ef8bd1157f325261ec1bbd05f125f031f28a2"
Oct 31 00:46:23.440554 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.440519    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"]
Oct 31 00:46:23.443128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.443101    4924 cpu_manager.go:414] "containerMap: removing container" podUID="6cf0b95f-fbed-4e37-b599-e164e93c17c9" containerName="revoke-keys"
Oct 31 00:46:23.443297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.443131    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cf0b95f-fbed-4e37-b599-e164e93c17c9" containerName="revoke-keys"
Oct 31 00:46:23.443411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.443345    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cf0b95f-fbed-4e37-b599-e164e93c17c9" containerName="revoke-keys"
Oct 31 00:46:23.447688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.447659    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.449717 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.449675    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"]
Oct 31 00:46:23.455064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.455032    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f99ac0\"/\"distinct-2-w0-f99ac0-kserve-self-signed-certs\""
Oct 31 00:46:23.579222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.579182    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.579403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.579236    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.579403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.579383    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.579516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.579465    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.579637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.579610    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2xk6\" (UniqueName: \"kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.579795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.579684    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.681403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.681364    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.681622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.681476    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.681622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.681511    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.681622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.681566    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.681622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.681622    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.681911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.681740    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2xk6\" (UniqueName: \"kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.682569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.682537    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.682865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.682817    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.683241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.683221    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.685516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.685493    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.709497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.708382    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.709497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.709159    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2xk6\" (UniqueName: \"kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6\") pod \"distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") " pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:23.761244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:23.761211    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:24.055442 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:24.055405    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"]
Oct 31 00:46:24.056015 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:46:24.055990    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e2d53f7_fe8b_4eec_8648_f3949ca77ca5.slice/crio-e8f8cb1a7d299987e850c6558f0a1c0a47fea9b6715408f1e0c92c70c360268a WatchSource:0}: Error finding container e8f8cb1a7d299987e850c6558f0a1c0a47fea9b6715408f1e0c92c70c360268a: Status 404 returned error can't find the container with id e8f8cb1a7d299987e850c6558f0a1c0a47fea9b6715408f1e0c92c70c360268a
Oct 31 00:46:24.493550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:24.493212    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" event={"ID":"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5","Type":"ContainerStarted","Data":"7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013"}
Oct 31 00:46:24.493550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:24.493265    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" event={"ID":"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5","Type":"ContainerStarted","Data":"e8f8cb1a7d299987e850c6558f0a1c0a47fea9b6715408f1e0c92c70c360268a"}
Oct 31 00:46:24.512008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:24.511944    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podStartSLOduration=1.511925695 podStartE2EDuration="1.511925695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:24.510124181 +0000 UTC m=+784.880907972" watchObservedRunningTime="2026-10-31 00:46:24.511925695 +0000 UTC m=+784.882709484"
Oct 31 00:46:24.761941 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:24.761837    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:28.409771 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:46:28.408395    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:46:28.642816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.642707    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"]
Oct 31 00:46:28.660153 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.660065    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.665918    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.677937    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.677991    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.678121    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.678154    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.678182    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.678220    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.678271    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kp7s\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.694786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.678372    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.740362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.740281    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:28.769771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.763187    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"]
Oct 31 00:46:28.773968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.773938    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.791948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.790850    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:28.795782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.795584    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.795782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.795730    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.795782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.795786    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.796081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.795943    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.796081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.796043    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.796081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.796071    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.796243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.796129    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.796243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.796177    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kp7s\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.796332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.796300    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.797617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.797345    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.797617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.797535    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.820520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.798142    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.820520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.817252    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.820520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.817389    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.855968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.855865    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"]
Oct 31 00:46:28.908780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.904831    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"]
Oct 31 00:46:28.909898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.909673    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.909898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.909794    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.909979    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.910029    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.910058    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6d8t\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.910169    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.910203    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.910254    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.910339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.910293    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:28.911256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.911191    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.919439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.919017    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.919439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.919373    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.939132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.938811    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kp7s\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:28.990790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.957853    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"]
Oct 31 00:46:28.995189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:28.995142    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.024873    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.025836    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029247    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029317    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029349    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029384    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029410    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029456    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029490    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029516    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029595    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029619    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029689    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.037839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.029803    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.050480    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.050622    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.050655    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6d8t\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.050692    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx9sp\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.050735    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.050793    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.051386    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.052901    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.053044    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.053600    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.070353    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.073564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.038203    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"]
Oct 31 00:46:29.075603 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.075231    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.116208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.106019    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.154361    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155245    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155291    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155345    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155414    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155446    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155510    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155598    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155713    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.155813    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx9sp\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.163398    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.163771    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.172299    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.173520    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.173843    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.174822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.174081    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.192246 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.192157    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx9sp\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.195425 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.195345    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.212908 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.212834    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6d8t\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.224800 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.224274    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.231479 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.229839    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:46:29.231479 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.230238    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:46:29.237403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.237322    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.272782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.272189    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.454832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.454014    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:29.455596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.455506    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:29.540147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.516994    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:29.543978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.542962    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:29.970104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:29.960838    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:46:30.380514 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.380399    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:30.480305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.427640    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:46:30.610212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.592666    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:46:30.619100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.619044    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:46:30.669605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.669505    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:30.719721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.711837    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:30.777770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.776700    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:30.777770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.776820    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:30.777770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.776874    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:30.919827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.919757    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:46:30.919827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.919798    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"]
Oct 31 00:46:30.920013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.919857    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:30.920013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.919892    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:46:30.920013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.919925    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:46:30.920013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.919972    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:46:30.920013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.920000    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:46:30.920244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.920029    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:46:30.920244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.920055    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:30.920244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.920079    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:46:30.920244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.920107    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:46:30.953770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:30.932330    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" event={"ID":"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819","Type":"ContainerStarted","Data":"b40a82424708379dbe1f84b7ec03b2cbbf99e0e537b211a297944cbcf57bd9a4"}
Oct 31 00:46:31.049510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.049463    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:31.049628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.049556    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:31.049628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.049603    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:31.100841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.100809    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"]
Oct 31 00:46:31.453792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.453726    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"]
Oct 31 00:46:31.463980 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.463094    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:31.463980 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.463195    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:31.463980 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.463238    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:31.948018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.947538    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" event={"ID":"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819","Type":"ContainerStarted","Data":"2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a"}
Oct 31 00:46:31.949885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.949856    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" event={"ID":"d25a8ae5-a217-4859-ac56-9ec49996c29a","Type":"ContainerStarted","Data":"75ee4f1be45a1682a8ea28bd5106b5756b48dd12eb457bf4eb3bb2d1f61fb70a"}
Oct 31 00:46:31.950013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.949906    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" event={"ID":"d25a8ae5-a217-4859-ac56-9ec49996c29a","Type":"ContainerStarted","Data":"dea90d14c6e8f17c6ed75df95e845dfb0d9906374016b2a400884e79006d9d48"}
Oct 31 00:46:31.951921 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.951889    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" event={"ID":"ec7ec421-602c-4c8d-9da2-1bf66b7deb55","Type":"ContainerStarted","Data":"90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15"}
Oct 31 00:46:31.952011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.951937    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" event={"ID":"ec7ec421-602c-4c8d-9da2-1bf66b7deb55","Type":"ContainerStarted","Data":"01bfca5fb46bea92fa69f7ae4b04c70edd7047e8209e77bcdc27b1fd0a979cbf"}
Oct 31 00:46:31.972013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.971950    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podStartSLOduration=4.97193125 podStartE2EDuration="4.97193125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:31.965400292 +0000 UTC m=+792.336184085" watchObservedRunningTime="2026-10-31 00:46:31.97193125 +0000 UTC m=+792.342715039"
Oct 31 00:46:31.985055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:31.984971    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podStartSLOduration=4.984953371 podStartE2EDuration="4.984953371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:31.984677168 +0000 UTC m=+792.355460977" watchObservedRunningTime="2026-10-31 00:46:31.984953371 +0000 UTC m=+792.355737159"
Oct 31 00:46:32.037770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.036990    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podStartSLOduration=4.036966525 podStartE2EDuration="4.036966525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:32.028107004 +0000 UTC m=+792.398890793" watchObservedRunningTime="2026-10-31 00:46:32.036966525 +0000 UTC m=+792.407750315"
Oct 31 00:46:32.071497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.071382    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:32.073499 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.073470    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:32.073583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.073529    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:32.458771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.458120    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:32.463770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.460105    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:32.463770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.460652    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:32.463770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.460700    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:32.463770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.461840    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:32.463770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:32.461881    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:33.081781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.081257    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:33.081781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.081323    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:33.461312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.461202    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:33.461573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.461544    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:33.462620 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.462595    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:33.463261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.463236    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:33.961769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.960468    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:33.991323 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:33.991295    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:34.125768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.125192    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:34.125768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.125252    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:34.457771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.456833    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:34.457771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.456896    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:34.457771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.457559    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:34.457771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.457597    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:34.693107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:34.692585    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:35.108933 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.108887    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:35.122903 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.122858    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:35.479935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.479454    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:35.479935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.479515    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:35.483770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.480589    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:35.483770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.480637    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:35.878050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.878012    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"]
Oct 31 00:46:35.923771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:35.902015    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.022772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.022271    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:36.075183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.074674    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:36.075183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.074738    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:36.086768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.084791    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:46:36.086768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.085578    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:46:36.127775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.127719    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"]
Oct 31 00:46:36.129459 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.129377    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.129732 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.129692    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w67xv\" (UniqueName: \"kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.130036 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.130019    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.160781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.157845    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:36.160781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.158293    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:46:36.168966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.168913    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:36.204795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.197516    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:36.267026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.243692    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.267026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.243882    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.267026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.243920    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w67xv\" (UniqueName: \"kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.267026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.245109    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.267026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.250314    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.290161 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.290091    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w67xv\" (UniqueName: \"kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv\") pod \"maas-api-revoke-sub-ef46cd9e-2c4gm\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.469827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.465877    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:36.469827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.465942    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:36.469827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.467095    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:36.469827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.467142    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:36.496815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.496785    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:36.731835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.727901    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:36.772772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.769831    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:36.776871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.776829    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:37.001833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:36.984908    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:37.038843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.037505    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:37.079772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.078972    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:37.079772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.079038    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:37.763856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.763312    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:37.763856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.763377    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:37.817835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.800950    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:37.817835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.801020    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:37.942021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.941973    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"]
Oct 31 00:46:37.942896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:37.942859    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm" event={"ID":"c3206258-f745-4418-8be2-24a13fee1dfb","Type":"ContainerStarted","Data":"ca72feb25df717ecc95df8c57ceac9b2eb66bf748e03a2feb7a69baa00fa90fd"}
Oct 31 00:46:38.072871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:38.072835    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 31 00:46:38.073048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:38.072932    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 31 00:46:38.455632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:38.455370    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Oct 31 00:46:38.455632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:38.455473    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Oct 31 00:46:38.456288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:38.456265    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:46:38.456485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:38.456459    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 31 00:46:39.049503 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.049453    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm" event={"ID":"c3206258-f745-4418-8be2-24a13fee1dfb","Type":"ContainerStarted","Data":"7932201f8142d9a2c01e405a4afb23fbcfaa48ff05b50da16c1a10e7ff15ac56"}
Oct 31 00:46:39.460159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.460063    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:39.460614 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.460583    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:39.461526 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.461503    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:46:39.484473 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.484398    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm" podStartSLOduration=4.484383319 podStartE2EDuration="4.484383319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:39.104458175 +0000 UTC m=+799.475241963" watchObservedRunningTime="2026-10-31 00:46:39.484383319 +0000 UTC m=+799.855167107"
Oct 31 00:46:39.515370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.515172    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:39.516028 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.516003    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:39.519149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.519119    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:46:39.889870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.889832    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:39.890774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.890739    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:39.891680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:39.891660    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:46:40.059222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:40.059185    4924 generic.go:356] "Generic (PLEG): container finished" podID="c3206258-f745-4418-8be2-24a13fee1dfb" containerID="7932201f8142d9a2c01e405a4afb23fbcfaa48ff05b50da16c1a10e7ff15ac56" exitCode=0
Oct 31 00:46:40.059641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:40.059282    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm" event={"ID":"c3206258-f745-4418-8be2-24a13fee1dfb","Type":"ContainerDied","Data":"7932201f8142d9a2c01e405a4afb23fbcfaa48ff05b50da16c1a10e7ff15ac56"}
Oct 31 00:46:41.255223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.255194    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:41.392400 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.392314    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca\") pod \"c3206258-f745-4418-8be2-24a13fee1dfb\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") "
Oct 31 00:46:41.392588 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.392567    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca\") pod \"c3206258-f745-4418-8be2-24a13fee1dfb\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") "
Oct 31 00:46:41.392670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.392636    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv\" (UniqueName: \"kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv\") pod \"c3206258-f745-4418-8be2-24a13fee1dfb\" (UID: \"c3206258-f745-4418-8be2-24a13fee1dfb\") "
Oct 31 00:46:41.392998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.392962    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca" pod "c3206258-f745-4418-8be2-24a13fee1dfb" (UID: "c3206258-f745-4418-8be2-24a13fee1dfb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:46:41.393271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.393256    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:41.395042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.395021    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca" pod "c3206258-f745-4418-8be2-24a13fee1dfb" (UID: "c3206258-f745-4418-8be2-24a13fee1dfb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:46:41.395111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.395053    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv" pod "c3206258-f745-4418-8be2-24a13fee1dfb" (UID: "c3206258-f745-4418-8be2-24a13fee1dfb"). InnerVolumeSpecName "kube-api-access-w67xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:46:41.494501 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.494465    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3206258-f745-4418-8be2-24a13fee1dfb-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:41.494501 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:41.494497    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w67xv\" (UniqueName: \"kubernetes.io/projected/c3206258-f745-4418-8be2-24a13fee1dfb-kube-api-access-w67xv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:42.078135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.078057    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"
Oct 31 00:46:42.078362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.078052    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm" event={"ID":"c3206258-f745-4418-8be2-24a13fee1dfb","Type":"ContainerDied","Data":"ca72feb25df717ecc95df8c57ceac9b2eb66bf748e03a2feb7a69baa00fa90fd"}
Oct 31 00:46:42.078362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.078195    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca72feb25df717ecc95df8c57ceac9b2eb66bf748e03a2feb7a69baa00fa90fd"
Oct 31 00:46:42.795228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.795189    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"]
Oct 31 00:46:42.796279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.796258    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c3206258-f745-4418-8be2-24a13fee1dfb" containerName="revoke-keys"
Oct 31 00:46:42.796332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.796283    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3206258-f745-4418-8be2-24a13fee1dfb" containerName="revoke-keys"
Oct 31 00:46:42.796491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.796480    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3206258-f745-4418-8be2-24a13fee1dfb" containerName="revoke-keys"
Oct 31 00:46:42.801101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.801079    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.824282 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.824249    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"]
Oct 31 00:46:42.924961 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.924924    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.924961 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.924965    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925179 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925089    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925252    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925328    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925482    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925512    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925688    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt6wt\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:42.925805 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:42.925788    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.027611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.027569    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.027827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.027625    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.027827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.027697    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.027966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.027843    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.027966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.027936    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.028069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.028019    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.028069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.028043    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.028069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.028064    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt6wt\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.028219 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.028096    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.028570 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.028542    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.029568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.029383    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.029568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.029481    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.031063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.030338    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.031063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.030432    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.031063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.030891    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"]
Oct 31 00:46:43.034590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.034457    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.040866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.040842    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.044277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.044237    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.047291 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.047232    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.059368 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.059198    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt6wt\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt\") pod \"e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") " pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.064639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.064607    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"]
Oct 31 00:46:43.124060 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.124022    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:43.129021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.128987    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129029    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129082    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129108    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129160    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129382 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129230    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129382 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129306    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpqw5\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129382 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129356    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.129576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.129557    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.216844 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.216697    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"]
Oct 31 00:46:43.224458 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.224424    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.230580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230551    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpqw5\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230596    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230704    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230729    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230764    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230798    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230817    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230841    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.230970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.230878    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.231213 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.231196    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.233064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.232113    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.233064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.232372    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.233064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.232426    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.233064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.232585    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.240757 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.240714    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.245878 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.244781    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"]
Oct 31 00:46:43.249442 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.248072    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.253425 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.253398    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.254071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.254047    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpqw5\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.332385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332289    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7npdn\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332345    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332417    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332455    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332499    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332515    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332568    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332596    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.332940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.332619    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.397558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.397523    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:43.433593 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433546    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.433858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433600    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.433858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433663    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.433858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433792    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.433858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433820    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.434120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433911    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.434120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433956    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.434120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.433993    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.434276 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.434242    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7npdn\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.435580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.435540    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.448576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.445185    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.448576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.445460    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.448576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.446547    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.449351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.449244    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.452561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.452530    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.452706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.452668    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.454215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.454161    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"]
Oct 31 00:46:43.454903 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:46:43.454878    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee1ad5b_ce6d_4946_a3f6_01a933e58d07.slice/crio-380e1592fd988f90bd4064323af7f1deb3a4f77de09ae1dc5e1f3be83b3a010c WatchSource:0}: Error finding container 380e1592fd988f90bd4064323af7f1deb3a4f77de09ae1dc5e1f3be83b3a010c: Status 404 returned error can't find the container with id 380e1592fd988f90bd4064323af7f1deb3a4f77de09ae1dc5e1f3be83b3a010c
Oct 31 00:46:43.459821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.459084    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.459821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.459172    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.459821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.459214    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.470005 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.469948    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7npdn\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.505056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.489317    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token\") pod \"e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") " pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.554809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.554734    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:43.724738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.720945    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.724738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.721020    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.724738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.721052    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.724738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.723787    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"]
Oct 31 00:46:43.881239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.881167    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"]
Oct 31 00:46:43.882111 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:46:43.882085    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6e0c4a_ba2a_492e_b7da_0bd5e24bb5a5.slice/crio-062cf78846ace9eafab358cba52819b310a1a4b3b31d9db06cd2b4e1452ef9ff WatchSource:0}: Error finding container 062cf78846ace9eafab358cba52819b310a1a4b3b31d9db06cd2b4e1452ef9ff: Status 404 returned error can't find the container with id 062cf78846ace9eafab358cba52819b310a1a4b3b31d9db06cd2b4e1452ef9ff
Oct 31 00:46:43.884522 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.884477    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.884610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.884573    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:43.884653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:43.884619    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:44.119899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.119862    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" event={"ID":"aee1ad5b-ce6d-4946-a3f6-01a933e58d07","Type":"ContainerStarted","Data":"56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42"}
Oct 31 00:46:44.127409 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.127381    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:44.129682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.129646    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" event={"ID":"aee1ad5b-ce6d-4946-a3f6-01a933e58d07","Type":"ContainerStarted","Data":"380e1592fd988f90bd4064323af7f1deb3a4f77de09ae1dc5e1f3be83b3a010c"}
Oct 31 00:46:44.129975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.129956    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" event={"ID":"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5","Type":"ContainerStarted","Data":"287bf48c0c475dababc89afbbf8cbdccfe36e327e6558ef4ece9dfb96f41adf5"}
Oct 31 00:46:44.137255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.130866    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" event={"ID":"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5","Type":"ContainerStarted","Data":"062cf78846ace9eafab358cba52819b310a1a4b3b31d9db06cd2b4e1452ef9ff"}
Oct 31 00:46:44.137255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.130904    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" event={"ID":"8677e702-1ca1-469c-bdd7-76e4f1512c9c","Type":"ContainerStarted","Data":"c83f09c67a49ae6a507156d614973b43d2801546d8729031df23056fe0c4ee81"}
Oct 31 00:46:44.137255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.130938    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" event={"ID":"8677e702-1ca1-469c-bdd7-76e4f1512c9c","Type":"ContainerStarted","Data":"b1a1dc900cda1226e3553ffa8bfd1606cf9f4f62c98007f8a5e457493ca225a1"}
Oct 31 00:46:44.137255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.125972    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:44.137255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.130999    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:44.147387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.147327    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podStartSLOduration=2.14730848 podStartE2EDuration="2.14730848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:44.14124829 +0000 UTC m=+804.512032078" watchObservedRunningTime="2026-10-31 00:46:44.14730848 +0000 UTC m=+804.518092271"
Oct 31 00:46:44.172405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.172332    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podStartSLOduration=2.172312434 podStartE2EDuration="2.172312434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:44.166781661 +0000 UTC m=+804.537565451" watchObservedRunningTime="2026-10-31 00:46:44.172312434 +0000 UTC m=+804.543096223"
Oct 31 00:46:44.191645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.191583    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podStartSLOduration=1.191563575 podStartE2EDuration="1.191563575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:44.188976657 +0000 UTC m=+804.559760446" watchObservedRunningTime="2026-10-31 00:46:44.191563575 +0000 UTC m=+804.562347346"
Oct 31 00:46:44.398724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.398621    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:44.400658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.400633    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:44.400757 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.400691    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:44.561866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.561832    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:44.569907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.569870    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:44.570253 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:44.570220    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:45.048769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.047819    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"]
Oct 31 00:46:45.125776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.100980    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"]
Oct 31 00:46:45.125776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.101130    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.151770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.127438    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:45.202773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.200710    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:45.202773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.148318    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5c2a03\"/\"distinct-w4-5c2a03-kserve-self-signed-certs\""
Oct 31 00:46:45.367645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.367539    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.367991 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.367961    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.368217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.368200    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.368370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.368354    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.368519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.368504    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrgc5\" (UniqueName: \"kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.368662 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.368649    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.407684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.407648    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:45.407987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.407958    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:45.472785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.472725    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.473083 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.473064    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.473284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.473267    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrgc5\" (UniqueName: \"kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.473456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.473442    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.473773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.473739    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.473985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.473967    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.486696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.486660    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.496478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.496439    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.497072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.497049    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.511331 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.511265    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrgc5\" (UniqueName: \"kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.516187 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.516121    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.517029 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.516985    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs\") pod \"distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") " pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:45.555611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.555184    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:45.555611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.555246    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:45.683824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:45.683051    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:46.287767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.287471    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:46.287767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.287544    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:46.321081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.290056    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:46:46.459910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.457946    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:46.459910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.458010    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:46.618045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.589856    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:46.618045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.589915    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:46.736781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.687631    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"]
Oct 31 00:46:46.815130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.780195    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:46.815130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.806858    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:46.980771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:46.977177    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:47.000389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.000351    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:47.144009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.143949    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:47.144628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.144231    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:47.286954 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.286891    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:47.316562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.316519    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:47.348775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.341529    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:46:47.409461 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.409428    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:47.409777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.409726    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:47.471441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.471401    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:47.559834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.556853    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:47.559834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:47.556912    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:48.124498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.124464    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:48.139379 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.139340    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:48.139948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.129596    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:46:48.140144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.140131    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"]
Oct 31 00:46:48.146842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.146807    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:46:48.451924 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.437482    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:48.451924 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.437547    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:48.451924 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.449244    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"]
Oct 31 00:46:48.451924 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.449303    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" event={"ID":"88b08800-ec7f-4d04-85a2-ef12a01a4c10","Type":"ContainerStarted","Data":"4bb5697ecfd04dfa004a174c19ca5b75b0049615137e94e7442eb250e480f216"}
Oct 31 00:46:48.451924 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.449406    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:48.464807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.462142    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:46:48.464807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.463178    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:46:48.646769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.643947    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:48.646769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.644205    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:48.646769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.644284    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxtth\" (UniqueName: \"kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:48.675712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.659880    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:48.675712 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.659946    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:48.678523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.678482    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:48.882974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.869444    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:46:49.120772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.967071    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:49.120980 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.988829    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.121235 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.121157    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.121310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.121258    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxtth\" (UniqueName: \"kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.132775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.078471    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:49.132775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:48.990486    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.156977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.003889    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:46:49.156977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.138806    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:49.156977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.138901    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:49.230773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.193386    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxtth\" (UniqueName: \"kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.276942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.228500    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:46:49.278844 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.278815    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:49.288419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.288388    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d2c8acf-wvtkq\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.350960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.350909    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" event={"ID":"88b08800-ec7f-4d04-85a2-ef12a01a4c10","Type":"ContainerStarted","Data":"f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4"}
Oct 31 00:46:49.514905 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.507190    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": context deadline exceeded"
Oct 31 00:46:49.550705 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.550669    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:49.551022 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.550987    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:49.565669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.565625    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:49.594024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.593891    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:49.630818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.630733    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podStartSLOduration=5.630708071 podStartE2EDuration="5.630708071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:49.486042687 +0000 UTC m=+809.856826481" watchObservedRunningTime="2026-10-31 00:46:49.630708071 +0000 UTC m=+810.001491861"
Oct 31 00:46:49.849988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.847605    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:49.849988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.847669    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:49.888297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:49.869432    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:46:50.074840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.070822    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:50.074840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.070957    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:46:50.075118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.074893    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:46:50.120820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.111863    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.123030    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.170908    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201421    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201052    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201598    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201079    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.200835    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201670    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201899    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:46:50.203815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.201951    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:46:50.300935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.300885    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:50.455371 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.455281    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:50.457861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.457820    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:50.584615 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.584568    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:50.585230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.585198    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:50.953772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.921309    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/health\": context deadline exceeded"
Oct 31 00:46:51.057604 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:50.978868    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:51.114509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.114465    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:51.186771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.185472    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:46:51.187702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.187681    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:46:51.193893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.193863    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:46:51.226772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.225509    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:51.238503 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.232321    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:51.238503 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.232374    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:51.562737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.562703    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:46:51.568977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.568932    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 31 00:46:51.572579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.572071    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:51.572579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.572133    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:51.574322 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.574017    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:51.663190 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.663158    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:46:51.974502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:51.974469    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:46:52.107793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:52.086874    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"]
Oct 31 00:46:52.220772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:52.200628    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:46:52.221094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:52.221055    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Oct 31 00:46:52.253102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:52.246222    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq" event={"ID":"fa190b51-33ce-47e2-b931-04d63781068a","Type":"ContainerStarted","Data":"8945cbeae3ef5f9aff236eeb477d8f917bfeb9814d74cbf5f9396fc4e78c03dc"}
Oct 31 00:46:52.559238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:52.559201    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:46:52.559867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:52.559817    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Oct 31 00:46:53.129109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.129063    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.130088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.130056    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.190770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.172584    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:53.747769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.745018    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.766862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.752518    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.766862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.754202    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.766862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.754281    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.766862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.764857    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:53.777889 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.777840    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:46:53.807877 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.807838    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:46:53.858802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.808254    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:53.858802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.812188    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:53.860049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.860013    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:46:53.860241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.860213    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:53.932429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.932392    4924 generic.go:356] "Generic (PLEG): container finished" podID="fa190b51-33ce-47e2-b931-04d63781068a" containerID="b2b25cfe506c4edfe0e4ddb4a01e703678ef8a357793c39772df61f8b5f40caf" exitCode=0
Oct 31 00:46:53.932829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:53.932797    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq" event={"ID":"fa190b51-33ce-47e2-b931-04d63781068a","Type":"ContainerDied","Data":"b2b25cfe506c4edfe0e4ddb4a01e703678ef8a357793c39772df61f8b5f40caf"}
Oct 31 00:46:54.134360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.134316    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:46:54.142789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.135786    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:46:54.158130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.143715    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:46:54.159041 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.158860    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:46:54.184163 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.184112    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:46:54.184571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.184462    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:54.298716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.298661    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:54.310777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.300170    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:46:54.331780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.316832    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:46:54.473883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.460296    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:46:54.601769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.576111    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:54.601769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.584461    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:54.620071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.610598    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:46:54.886768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.861728    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:46:54.886768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.862039    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:54.894742 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.894701    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:54.895276 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.895248    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:54.896152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.896126    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:46:54.906508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.906477    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:54.906970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.906955    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:46:54.907114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.907105    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:46:54.907219 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.907211    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:46:54.907336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.907326    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:46:54.907431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.907423    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:54.912956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.909435    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:46:54.912956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.909492    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:54.957383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.944457    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:54.983958 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.983924    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:46:54.985697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:54.985665    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:46:55.321828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.321782    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:46:55.409785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.406109    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:46:55.431642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.431614    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:46:55.462433 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.462402    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:55.467910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.467454    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:55.540449 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.540403    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:46:55.562789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.562690    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:46:55.586340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.586259    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:46:55.663685 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.663652    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:46:55.696429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.696396    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:55.723404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.713441    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded"
Oct 31 00:46:55.723404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.718299    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:46:55.724945 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.724704    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:55.904882 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.904857    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:55.998294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.998261    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"
Oct 31 00:46:55.998516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.998290    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq" event={"ID":"fa190b51-33ce-47e2-b931-04d63781068a","Type":"ContainerDied","Data":"8945cbeae3ef5f9aff236eeb477d8f917bfeb9814d74cbf5f9396fc4e78c03dc"}
Oct 31 00:46:55.999636 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:55.999605    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8945cbeae3ef5f9aff236eeb477d8f917bfeb9814d74cbf5f9396fc4e78c03dc"
Oct 31 00:46:56.017969 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.017918    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:46:56.065518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.065480    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth\" (UniqueName: \"kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth\") pod \"fa190b51-33ce-47e2-b931-04d63781068a\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") "
Oct 31 00:46:56.065725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.065665    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca\") pod \"fa190b51-33ce-47e2-b931-04d63781068a\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") "
Oct 31 00:46:56.065895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.065874    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca\") pod \"fa190b51-33ce-47e2-b931-04d63781068a\" (UID: \"fa190b51-33ce-47e2-b931-04d63781068a\") "
Oct 31 00:46:56.066216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.066189    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca" pod "fa190b51-33ce-47e2-b931-04d63781068a" (UID: "fa190b51-33ce-47e2-b931-04d63781068a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:46:56.066523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.066505    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:56.067810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.067730    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth" pod "fa190b51-33ce-47e2-b931-04d63781068a" (UID: "fa190b51-33ce-47e2-b931-04d63781068a"). InnerVolumeSpecName "kube-api-access-sxtth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:46:56.070822 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.070800    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca" pod "fa190b51-33ce-47e2-b931-04d63781068a" (UID: "fa190b51-33ce-47e2-b931-04d63781068a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:46:56.169216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.169135    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxtth\" (UniqueName: \"kubernetes.io/projected/fa190b51-33ce-47e2-b931-04d63781068a-kube-api-access-sxtth\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:56.169419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:56.169393    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa190b51-33ce-47e2-b931-04d63781068a-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:46:57.858662 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.858624    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"]
Oct 31 00:46:57.859722 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.859697    4924 cpu_manager.go:414] "containerMap: removing container" podUID="fa190b51-33ce-47e2-b931-04d63781068a" containerName="revoke-keys"
Oct 31 00:46:57.859910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.859893    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa190b51-33ce-47e2-b931-04d63781068a" containerName="revoke-keys"
Oct 31 00:46:57.863997 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.863963    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa190b51-33ce-47e2-b931-04d63781068a" containerName="revoke-keys"
Oct 31 00:46:57.865060 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:46:57.865028    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:46:57.876681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.876641    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"]
Oct 31 00:46:57.876876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.876814    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.907787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.907704    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.907966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.907828    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.907966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.907893    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.908130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.908017    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.908130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.908054    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.908130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.908100    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76489\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.908335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.908207    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.908335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.908233    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:57.908335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:57.908270    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009052    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009135    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009160    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009187    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76489\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009252    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009270    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009296    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009378    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009406    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.009549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.009538    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.010404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.010374    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.010740 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.010711    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.010857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.010809    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.010857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.010809    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.012178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.012153    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.012536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.012516    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.017196 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.017173    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.017481 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.017462    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76489\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.208718 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.208622    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:58.520660 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:46:58.520628    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6643d43_f92c_4820_b206_cd6e78a7c3b5.slice/crio-57d10b7fa595ff3905ae1008bc836ade9e988b4f224e175d37e7b89515d85dea WatchSource:0}: Error finding container 57d10b7fa595ff3905ae1008bc836ade9e988b4f224e175d37e7b89515d85dea: Status 404 returned error can't find the container with id 57d10b7fa595ff3905ae1008bc836ade9e988b4f224e175d37e7b89515d85dea
Oct 31 00:46:58.521363 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.521340    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"]
Oct 31 00:46:58.523617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.523582    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:58.523709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.523648    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:58.523709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:58.523677    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:46:59.065445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:59.065406    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" event={"ID":"e6643d43-f92c-4820-b206-cd6e78a7c3b5","Type":"ContainerStarted","Data":"03e1d1baa5edf4db57e9f49937e8639a4ceb744134e88e7424880516e0fc41c1"}
Oct 31 00:46:59.066004 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:59.065473    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" event={"ID":"e6643d43-f92c-4820-b206-cd6e78a7c3b5","Type":"ContainerStarted","Data":"57d10b7fa595ff3905ae1008bc836ade9e988b4f224e175d37e7b89515d85dea"}
Oct 31 00:46:59.095067 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:59.092387    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podStartSLOduration=2.092366492 podStartE2EDuration="2.092366492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:59.085224204 +0000 UTC m=+819.456008006" watchObservedRunningTime="2026-10-31 00:46:59.092366492 +0000 UTC m=+819.463150281"
Oct 31 00:46:59.214772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:59.214502    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:46:59.214772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:59.214634    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:46:59.214772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:46:59.214677    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:00.213779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:00.213147    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:47:00.213779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:00.213213    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:01.211865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:01.211825    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:47:01.212179 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:01.212147    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:02.220771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.220170    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:47:02.220771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.220236    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:02.747927 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.747878    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"]
Oct 31 00:47:02.760146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.759358    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.760146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.759173    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"]
Oct 31 00:47:02.762693 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.762662    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-b433e6\"/\"openshift-service-ca.crt\""
Oct 31 00:47:02.763626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.763600    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-b433e6\"/\"kube-root-ca.crt\""
Oct 31 00:47:02.763809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.763785    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-b433e6\"/\"distinct-2-w2-b433e6-kserve-self-signed-certs\""
Oct 31 00:47:02.763987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.763971    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-b433e6\"/\"default-dockercfg-8qm9p\""
Oct 31 00:47:02.804772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.801909    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswhl\" (UniqueName: \"kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.804772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.802064    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.804772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.802144    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.804772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.802207    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.804772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.802237    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.804772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.802371    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.974248 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.974207    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswhl\" (UniqueName: \"kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.974395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.974347    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.974460 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.974398    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.974494 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.974463    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.974531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.974493    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.974561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.974553    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.976069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.976036    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.996768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.976848    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.996768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.977473    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.996768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.988168    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:02.996768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:02.991995    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:03.065522 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:03.065490    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswhl\" (UniqueName: \"kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl\") pod \"distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") " pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:03.080157 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:03.080122    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:03.270678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:03.270206    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:47:03.270678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:03.270287    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:03.441771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:03.441654    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"]
Oct 31 00:47:04.153564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:04.153526    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" event={"ID":"2e29f410-c8ad-44ca-98a2-df59935478ef","Type":"ContainerStarted","Data":"918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce"}
Oct 31 00:47:04.153840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:04.153576    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" event={"ID":"2e29f410-c8ad-44ca-98a2-df59935478ef","Type":"ContainerStarted","Data":"a1f291715194ffbc9d39d4f9674fa60606fdd115ad1d70dcb0526772f9d5f77b"}
Oct 31 00:47:04.174328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:04.174264    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podStartSLOduration=2.174248651 podStartE2EDuration="2.174248651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:04.170838326 +0000 UTC m=+824.541622131" watchObservedRunningTime="2026-10-31 00:47:04.174248651 +0000 UTC m=+824.545032464"
Oct 31 00:47:04.212929 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:04.212885    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:47:04.213244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:04.213214    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:05.080819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:05.080785    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:05.210078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:05.210037    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 31 00:47:05.210294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:05.210114    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 31 00:47:06.214472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:06.214436    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:47:06.215005 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:06.214956    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:47:06.215864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:06.215843    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:47:10.661825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:10.661792    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"]
Oct 31 00:47:10.690773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:10.670713    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:10.693984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:10.692116    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:47:10.693984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:10.692467    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:47:10.718525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:10.704089    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"]
Oct 31 00:47:11.004961 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:10.813723    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g98rw\" (UniqueName: \"kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.236599 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.214905    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.265311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.265219    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.359413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.359367    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.438414 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.417065    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.450173 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.439639    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.457127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.457090    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g98rw\" (UniqueName: \"kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.457506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.457485    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.457778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.457735    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.465899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.465850    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.466478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.466446    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:11.523831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.499887    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.560772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.544849    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.560772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.555062    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g98rw\" (UniqueName: \"kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.592923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.592874    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.595653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.595624    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dc73a80c-dvgcn\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:11.684711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.684661    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:11.735737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.734343    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:11.763868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:11.763591    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:12.008097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.008056    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:12.283838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.257550    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:12.283838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.258843    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:12.448362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.445844    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:47:12.448362 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.446665    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:47:12.473894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.473846    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:12.474596 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.474541    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:12.475102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.475084    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:47:12.475277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.475264    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:47:12.478878 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.478845    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:12.479394 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.479369    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:47:12.479798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.479737    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:47:12.479936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.479926    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:12.545504 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.545393    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:12.550771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.545886    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:12.669318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.597822    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:12.669318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.598226    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:12.751801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.690422    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:12.751801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.690872    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:47:12.787802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.778513    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:47:12.787802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:12.786088    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:47:13.013926 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.013831    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.014450 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.014426    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:13.507916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.469226    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:13.507916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.506175    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:13.507916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.470697    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.507916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.506295    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.507916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.470664    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.586798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.581916    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:13.586798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.582013    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.592090 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.592048    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.691669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.641603    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:13.691960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.691944    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:13.718172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.692301    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:13.726943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.688293    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.726943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.691247    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:13.788411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:13.788368    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:47:14.062667 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.062566    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:14.492005 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.491948    4924 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/health\": context deadline exceeded"
Oct 31 00:47:14.569916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.569860    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:14.626611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.626549    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:14.643078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.643032    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:14.659379 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.659268    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:14.664867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.664827    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:14.719801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.691928    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:14.755770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.731103    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.119816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.782426    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.364664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.783002    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.380003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.783033    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:47:15.380427 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.783062    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.380580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:14.866501    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.411038 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:15.230714    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/health\": context deadline exceeded"
Oct 31 00:47:15.411595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:15.325031    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.675522 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:15.675422    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:47:15.693215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:15.693168    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.780019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:15.661131    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:15.965880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:15.940959    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4" podUID="42092e39-d9c6-47f8-b6dd-b4921bdc955b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.035883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.035821    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.058929 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.056891    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.060048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.058158    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.132004 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.058260    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.188787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.183875    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 31 00:47:16.255958 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.242988    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.376806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.369012    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:16.376806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.375883    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Oct 31 00:47:16.439119 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.432862    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:47:16.470766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.469207    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:16.550653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.548317    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:47:16.576874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.564428    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:16.576874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.566424    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:16.589524 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.589491    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:47:16.618490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.618453    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:16.640861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.640817    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn" event={"ID":"54bac04b-501a-4acb-bb7b-6ee4f2f483a6","Type":"ContainerStarted","Data":"94dd8cc523d5430d8976fb1f525152c580efd138849d9e8b3f75a6cfd823649a"}
Oct 31 00:47:16.648299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.648267    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:47:16.692540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.692500    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"]
Oct 31 00:47:16.724002 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.723971    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"]
Oct 31 00:47:16.724256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.724241    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"]
Oct 31 00:47:16.725000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.724975    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.750048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.749992    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"]
Oct 31 00:47:16.751618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.751112    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.757008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.756981    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:16.757250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.757229    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:47:16.757250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.757041    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.757456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.757323    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:16.757456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.757354    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:47:16.757456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.757371    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"]
Oct 31 00:47:16.757456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.757401    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"]
Oct 31 00:47:16.758671 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.758652    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:16.758821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.758699    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:16.758821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.758719    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"]
Oct 31 00:47:16.763438 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.763392    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"]
Oct 31 00:47:16.763938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.763920    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.769149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.769124    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"]
Oct 31 00:47:16.769292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.769227    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.775301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.775271    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"]
Oct 31 00:47:16.818792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.816017    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"]
Oct 31 00:47:16.848694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.848658    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"]
Oct 31 00:47:16.854167 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.854135    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:16.857736 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.857683    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"]
Oct 31 00:47:16.867907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.867875    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.868192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868164    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvsmv\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.868302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868290    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.868356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868319    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6blg5\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.868398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868365    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.868444 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868395    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.868489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868440    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ccnd\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.868489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868483    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.868578 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868511    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.868578 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868537    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.868578 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868559    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.868706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868606    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.868706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868649    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.868706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868693    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.868883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868758    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.868883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868794    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.868883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868816    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.868883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868866    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.869053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868895    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.869053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.868924    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.869053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869012    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.869053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869038    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.869229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869061    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.869229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869089    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.869229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869138    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.869229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869182    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.869229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869215    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.869463 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869263    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.869463 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869293    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.869463 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869370    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.869463 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869409    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.869672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869481    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.869672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869506    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.869672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869574    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.869914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.869810    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.871799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.871778    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.871944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.871930    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnpf4\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.872463 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.872443    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.872654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.872636    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.872843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.872817    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.873328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.873309    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.873469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.873454    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.873595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.873580    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.878630 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.878600    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.879116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.879093    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.981136 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981084    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.981136 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981139    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981164    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981229    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981259    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981277    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981293    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981319    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981350    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981375    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981401    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.981403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981419    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981441    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981484    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981527    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981554    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981599    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981628    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981691    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981739    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnpf4\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981814    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981844    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.981961    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982025    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.982058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982053    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982086    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982120    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982123    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982151    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982185    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982245    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982326    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvsmv\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982366    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982433    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982464    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982460    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6blg5\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982571    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982608    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982670    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ccnd\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982709    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982757    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982792    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982796    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982820    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982896    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982962    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.982994    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.983037    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.983075    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.983099    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.983122    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.983265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.983140    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cffw\" (UniqueName: \"kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:16.984531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.983396    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.984531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.984253    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.987850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.987816    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.988600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.988575    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:16.991120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.989392    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:16.991120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.990092    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:16.991120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.990346    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:16.991120 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.991034    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:16.992678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.992636    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.992822    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.993396    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.994988    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.995453    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.995856    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.996072    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.996771    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.996838    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.997836    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:16.999241    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.001163    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.001562    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.002081    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.002319    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.002492    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.003066    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.005520    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.006334    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.007369    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.007415    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.009311    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.011217    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.023766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.016268    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.028602    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.029224    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ccnd\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.033314    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token\") pod \"e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") " pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.033314    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.033995    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvsmv\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.034052    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.043220 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.039644    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6blg5\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.050175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.049921    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnpf4\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4\") pod \"e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") " pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.050405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.050380    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token\") pod \"e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") " pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.052542 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.052479    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:17.090513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.089976    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cffw\" (UniqueName: \"kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.090513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.090226    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.090513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.090301    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.091144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.091120    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.093644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.093624    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.110137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.110005    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:17.111064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.111003    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cffw\" (UniqueName: \"kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw\") pod \"maas-api-revoke-sub-ca9535c5-qpqgt\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.126870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.126621    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:17.136967 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.136809    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:17.151147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.148872    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:17.185103 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.175268    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:17.382851 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.382730    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"]
Oct 31 00:47:17.468281 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:17.468234    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847e6d33_4e12_49e7_96c3_7f2bc41300fc.slice/crio-d0ae581692532dee2583a76626697ec729d6ea094484ac6e76cd25952360eff5 WatchSource:0}: Error finding container d0ae581692532dee2583a76626697ec729d6ea094484ac6e76cd25952360eff5: Status 404 returned error can't find the container with id d0ae581692532dee2583a76626697ec729d6ea094484ac6e76cd25952360eff5
Oct 31 00:47:17.476130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.475549    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.476130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.475645    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.476130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.475686    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.516047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.516006    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"]
Oct 31 00:47:17.520372 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:17.520340    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8be0b53_3a6f_4b9f_8cb8_4c24179eca02.slice/crio-5423f79a252579fbad69caed3b97ba5cc8a9a2ca9f4660c87d10d79718b75928 WatchSource:0}: Error finding container 5423f79a252579fbad69caed3b97ba5cc8a9a2ca9f4660c87d10d79718b75928: Status 404 returned error can't find the container with id 5423f79a252579fbad69caed3b97ba5cc8a9a2ca9f4660c87d10d79718b75928
Oct 31 00:47:17.523842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.523795    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.528159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.528119    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.528350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.528328    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.556039 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:17.556005    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1eab11_d772_43c7_94ac_8a58d445ce90.slice/crio-51a6ee5d27f0c005366ffc5a0e210793b92b93cbc89696a515f4bd107f89c4df WatchSource:0}: Error finding container 51a6ee5d27f0c005366ffc5a0e210793b92b93cbc89696a515f4bd107f89c4df: Status 404 returned error can't find the container with id 51a6ee5d27f0c005366ffc5a0e210793b92b93cbc89696a515f4bd107f89c4df
Oct 31 00:47:17.560566 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.560522    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.560692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.560674    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.560795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.560759    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"]
Oct 31 00:47:17.560866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.560736    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.657310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.656696    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"]
Oct 31 00:47:17.663940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.663209    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.663940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.663297    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.663940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.663340    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.671884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.671852    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" event={"ID":"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02","Type":"ContainerStarted","Data":"5423f79a252579fbad69caed3b97ba5cc8a9a2ca9f4660c87d10d79718b75928"}
Oct 31 00:47:17.674254 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.674225    4924 generic.go:356] "Generic (PLEG): container finished" podID="54bac04b-501a-4acb-bb7b-6ee4f2f483a6" containerID="8fc0db991dfd2e7e7ba092966f1fd5785876ef419eab5f8f86405d2589c07e9a" exitCode=0
Oct 31 00:47:17.674611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.674592    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn" event={"ID":"54bac04b-501a-4acb-bb7b-6ee4f2f483a6","Type":"ContainerDied","Data":"8fc0db991dfd2e7e7ba092966f1fd5785876ef419eab5f8f86405d2589c07e9a"}
Oct 31 00:47:17.680246 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.680217    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" event={"ID":"aa1eab11-d772-43c7-94ac-8a58d445ce90","Type":"ContainerStarted","Data":"51a6ee5d27f0c005366ffc5a0e210793b92b93cbc89696a515f4bd107f89c4df"}
Oct 31 00:47:17.685064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.683609    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" event={"ID":"847e6d33-4e12-49e7-96c3-7f2bc41300fc","Type":"ContainerStarted","Data":"d1af9c4211e5eaf2470891324ab134516bc614785cfb3b2a9e6d79e9103c20e5"}
Oct 31 00:47:17.685064 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.683664    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" event={"ID":"847e6d33-4e12-49e7-96c3-7f2bc41300fc","Type":"ContainerStarted","Data":"d0ae581692532dee2583a76626697ec729d6ea094484ac6e76cd25952360eff5"}
Oct 31 00:47:17.879987 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:17.876318    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e5244c_8ea2_454f_a1fe_ca4333aeba6a.slice/crio-9659e888beb18ebab407997e24208d2413a07c282e6ca5972bfd0c3abc282acb WatchSource:0}: Error finding container 9659e888beb18ebab407997e24208d2413a07c282e6ca5972bfd0c3abc282acb: Status 404 returned error can't find the container with id 9659e888beb18ebab407997e24208d2413a07c282e6ca5972bfd0c3abc282acb
Oct 31 00:47:17.949210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.949130    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.949370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.949277    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.949431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.949368    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:47:17.973159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.968395    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"]
Oct 31 00:47:17.973159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.970913    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"]
Oct 31 00:47:17.979031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:17.978969    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podStartSLOduration=5.978946295 podStartE2EDuration="5.978946295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:17.761941616 +0000 UTC m=+838.132725406" watchObservedRunningTime="2026-10-31 00:47:17.978946295 +0000 UTC m=+838.349730083"
Oct 31 00:47:18.054867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.054782    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:18.055056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.054898    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:18.055056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.054962    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:18.698850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.697949    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" event={"ID":"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02","Type":"ContainerStarted","Data":"fdbe9b9d116b9a5bd6e71fdf728dce9db4be3bc5ea9e1889b768296c454d7da2"}
Oct 31 00:47:18.700398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.700058    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" event={"ID":"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb","Type":"ContainerStarted","Data":"7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea"}
Oct 31 00:47:18.700398 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.700118    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" event={"ID":"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb","Type":"ContainerStarted","Data":"63e5e34cb65e020ffada570267b4191dece367761e09abea7f507e4901162341"}
Oct 31 00:47:18.702734 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.702702    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" event={"ID":"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b","Type":"ContainerStarted","Data":"b0c693e04f0ad1ff8ca481a92504f38db23a0253ed30d38907611b87aaee38bc"}
Oct 31 00:47:18.702881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.702767    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" event={"ID":"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b","Type":"ContainerStarted","Data":"3944ef19f4edb1b94b3803df484f417b844e28f616c905d9df457a79eeefe698"}
Oct 31 00:47:18.704725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.704696    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" event={"ID":"aa1eab11-d772-43c7-94ac-8a58d445ce90","Type":"ContainerStarted","Data":"2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10"}
Oct 31 00:47:18.707113 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.707093    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt" event={"ID":"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a","Type":"ContainerStarted","Data":"176aa9d00a4b90d2a25826515a72252c12aba9c2fd33fb9ef3910506b7b024b5"}
Oct 31 00:47:18.707257 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.707244    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt" event={"ID":"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a","Type":"ContainerStarted","Data":"9659e888beb18ebab407997e24208d2413a07c282e6ca5972bfd0c3abc282acb"}
Oct 31 00:47:18.725780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.724185    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podStartSLOduration=5.724166743 podStartE2EDuration="5.724166743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:18.71669559 +0000 UTC m=+839.087479401" watchObservedRunningTime="2026-10-31 00:47:18.724166743 +0000 UTC m=+839.094950532"
Oct 31 00:47:18.741033 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.740968    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podStartSLOduration=5.740947961 podStartE2EDuration="5.740947961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:18.735558724 +0000 UTC m=+839.106342522" watchObservedRunningTime="2026-10-31 00:47:18.740947961 +0000 UTC m=+839.111731795"
Oct 31 00:47:18.762169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.762088    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt" podStartSLOduration=3.762072863 podStartE2EDuration="3.762072863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:18.752129749 +0000 UTC m=+839.122913538" watchObservedRunningTime="2026-10-31 00:47:18.762072863 +0000 UTC m=+839.132856654"
Oct 31 00:47:18.788182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.788121    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podStartSLOduration=5.78810419 podStartE2EDuration="5.78810419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:18.783304585 +0000 UTC m=+839.154088376" watchObservedRunningTime="2026-10-31 00:47:18.78810419 +0000 UTC m=+839.158887978"
Oct 31 00:47:18.844657 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:18.844595    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podStartSLOduration=5.84457688 podStartE2EDuration="5.84457688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:18.814035131 +0000 UTC m=+839.184818922" watchObservedRunningTime="2026-10-31 00:47:18.84457688 +0000 UTC m=+839.215360669"
Oct 31 00:47:19.092116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.091950    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:19.092116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.092039    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:19.138098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.138060    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:19.155936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.155901    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:19.156149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.156136    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:19.156270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.156259    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:19.156446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.138895    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:19.167777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.156549    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:19.168508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.165607    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:19.168508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.140762    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:19.168508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.142847    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:19.168508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.168365    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:19.168508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.168421    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:19.168508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.168468    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:19.723776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:19.718132    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:20.073767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.073468    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:20.073767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.073532    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:20.188283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.159587    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded"
Oct 31 00:47:20.364266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.364164    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:20.364584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.364541    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:20.366998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.365237    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca\") pod \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") "
Oct 31 00:47:20.399862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.397847    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.417066    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.417124    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.417195    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.417219    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.417271    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.417289    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:20.462818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.418929    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca" pod "54bac04b-501a-4acb-bb7b-6ee4f2f483a6" (UID: "54bac04b-501a-4acb-bb7b-6ee4f2f483a6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:20.464944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.464892    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:20.503770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.503101    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:20.518774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.508092    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:20.556359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.528606    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:20.556359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.546459    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:20.660277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.660146    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:20.672905 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.672848    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:20.695625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.695583    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:20.715780 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.715729    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca\") pod \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") "
Oct 31 00:47:20.742545 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.742505    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw\" (UniqueName: \"kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw\") pod \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\" (UID: \"54bac04b-501a-4acb-bb7b-6ee4f2f483a6\") "
Oct 31 00:47:21.081108 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:20.739046    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca" pod "54bac04b-501a-4acb-bb7b-6ee4f2f483a6" (UID: "54bac04b-501a-4acb-bb7b-6ee4f2f483a6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:21.081108 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.050843    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/health\": context deadline exceeded"
Oct 31 00:47:21.081108 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.050898    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:21.164832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.068838    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:21.164832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.163675    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:21.189262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.177821    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw" pod "54bac04b-501a-4acb-bb7b-6ee4f2f483a6" (UID: "54bac04b-501a-4acb-bb7b-6ee4f2f483a6"). InnerVolumeSpecName "kube-api-access-g98rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:21.189262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.178128    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:21.189262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.178195    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:21.189262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.178867    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:21.189262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.178928    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:21.229920 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.229864    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:21.359828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.356915    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:21.359828 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:47:21.301855    4924 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.377s"
Oct 31 00:47:21.406769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.287330    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:21.406769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.381811    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:21.406769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.356712    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:21.406769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.381984    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:21.406769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.404276    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:21.406769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.404330    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g98rw\" (UniqueName: \"kubernetes.io/projected/54bac04b-501a-4acb-bb7b-6ee4f2f483a6-kube-api-access-g98rw\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:21.438773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.413866    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:21.464973 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.464938    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:21.559809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.483012    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:21.860861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.571340    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:22.663078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.571632    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:47:22.663626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.717313    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:23.082201 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:21.974218    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:23.100049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.274854    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:23.103987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.329866    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:47:23.110001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.330398    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:23.114008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.435302    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.135858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.524053    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:47:23.136308 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.474827    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.158777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.524204    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.159114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.159077    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:23.159413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.680337    4924 generic.go:356] "Generic (PLEG): container finished" podID="b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" containerID="176aa9d00a4b90d2a25826515a72252c12aba9c2fd33fb9ef3910506b7b024b5" exitCode=0
Oct 31 00:47:23.159573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.780212    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:23.313970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.781384    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:23.314434 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.723545    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:23.314739 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.757760    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:23.315159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.595083    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.315323 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.315294    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:23.315429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.210973    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:23.315528 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.211103    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:23.315616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.780171    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:23.315948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:22.759213    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.316084 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.316062    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:23.358897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.354404    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.414078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.360060    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:23.553333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.553297    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.553605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.553561    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:23.570821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.565506    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.570821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.565564    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:23.570821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.565634    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.570821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.565656    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:23.580300 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.580266    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:23.580496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.580321    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:23.671881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.664549    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:23.676174 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.676131    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"
Oct 31 00:47:23.923548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.923376    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:23.923719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:23.923555    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt" event={"ID":"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a","Type":"ContainerDied","Data":"176aa9d00a4b90d2a25826515a72252c12aba9c2fd33fb9ef3910506b7b024b5"}
Oct 31 00:47:24.091889 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.091837    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.195173 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.114102    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.196327 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.114142    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.220528 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.139034    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.220528 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.213858    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.270240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.270200    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:24.270518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.270489    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:24.271523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.271497    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:24.271697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.271676    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:24.271873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.271856    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:24.271991 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.271972    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:24.272191 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.272175    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:24.272312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.272293    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:24.272812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.272732    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:24.272965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.272944    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:24.332126 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.316692    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.332126 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.323378    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:24.350955 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.350849    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:24.351568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.351521    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.354167 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.354122    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:24.371568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.371521    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.428379    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn" event={"ID":"54bac04b-501a-4acb-bb7b-6ee4f2f483a6","Type":"ContainerDied","Data":"94dd8cc523d5430d8976fb1f525152c580efd138849d9e8b3f75a6cfd823649a"}
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.428547    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94dd8cc523d5430d8976fb1f525152c580efd138849d9e8b3f75a6cfd823649a"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.428706    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.428857    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.428900    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.429037    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.429158    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:47:24.429771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.429192    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:47:24.468591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.468502    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:24.516774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.500186    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:47:24.518308 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.518279    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:24.540982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.540942    4924 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.687773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.671214    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:24.883882 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:24.883693    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.049881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.049829    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.163177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.163062    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:25.195842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.195789    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:25.196089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.195511    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.196506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.166327    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:25.196686 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.196665    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:25.196877 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.196841    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:47:25.287036 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.166374    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:25.313193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.203886    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.313517 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.313486    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:25.313641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.280519    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.355582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.256875    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.356311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.193087    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:25.356471 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.356448    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:25.356582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.351897    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.356933 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.351852    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.357262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.256642    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:25.357400 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.357380    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:25.357493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.355165    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.357833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.351918    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.379470 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.355119    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.380970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.379077    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:25.705791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.696365    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:47:25.720294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:25.720245    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.077669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.077623    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 31 00:47:26.167790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.151617    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:26.167790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.151680    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:26.167790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.159821    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:26.167790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.159888    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:26.216617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.216560    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.217411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.217374    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.252730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.226414    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:26.252730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.226470    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:26.252730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.245904    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:26.252730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.245968    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:26.272091 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.272048    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.320251 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.320010    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.321093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.321063    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.372914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.356800    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.688827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.357906    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:47:26.698015 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.362274    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:26.706603 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.486305    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:47:26.721790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.488839    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.752336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.496206    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:47:26.754163 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.754107    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:26.781847 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.496300    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:26.814048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.813995    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:26.922021 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.921963    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:26.922338 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.496274    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:27.006969 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:26.956909    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": context deadline exceeded"
Oct 31 00:47:27.182157 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.160915    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.245897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.243326    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.302883    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.264130    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.311864    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.264192    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.311937    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.311060    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.311989    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.311264    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:27.317770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.312043    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:27.364772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.320823    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:47:27.701291 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.697260    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.812806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.701603    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.812806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.727065    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.812806 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.727146    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:47:27.817795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.756939    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:27.817795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.757112    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:47:27.852802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.851793    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:27.895015 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:27.870833    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:28.165774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.151495    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:28.165774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.151588    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:28.165774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.153439    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.166094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.165866    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:28.166094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.165921    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:28.166094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.166023    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:28.166094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.166061    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:28.166797 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.166765    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:28.166926 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.166829    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:28.175416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.175379    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:28.175669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.175635    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:28.176367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.176345    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:28.176519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.176493    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:28.275364 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.275323    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/health\": context deadline exceeded"
Oct 31 00:47:28.366774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.334956    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:28.366774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.335533    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.426963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.426876    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.679319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.679220    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:28.890932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.890856    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:28.891383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.891346    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.891739 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.891710    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.892124 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.892080    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.948974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.892269    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.948974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.892311    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.948974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.892331    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:28.948974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:28.892420    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.206464 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.206362    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:29.206708 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.206675    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:29.378801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.360912    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.379473 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:47:29.379426    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:47:29.379582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.379490    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:29.379582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.379532    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:29.380905 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.380875    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:47:29.381429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.381405    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:29.446199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.446132    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:29.446460 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.445541    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca\") pod \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") "
Oct 31 00:47:29.446723 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.446039    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:29.446901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.446879    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:29.474859 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.474031    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca" pod "b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" (UID: "b1e5244c-8ea2-454f-a1fe-ca4333aeba6a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:29.497992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.476439    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:29.497992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.494520    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:29.497992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.494579    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:29.708456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.688790    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca\") pod \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") "
Oct 31 00:47:29.708456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.689145    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw\" (UniqueName: \"kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw\") pod \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\" (UID: \"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a\") "
Oct 31 00:47:29.802392 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.802329    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.839801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.835674    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.854981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.854913    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.913754 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.913673    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.928781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.927887    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw" pod "b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" (UID: "b1e5244c-8ea2-454f-a1fe-ca4333aeba6a"). InnerVolumeSpecName "kube-api-access-6cffw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:29.946015 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.945973    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.946363 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.946336    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:47:29.955508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.951871    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.996940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.988570    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:47:29.996940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.988662    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:29.996940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.988705    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:30.011768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:29.998840    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cffw\" (UniqueName: \"kubernetes.io/projected/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-kube-api-access-6cffw\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:30.012015 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.002540    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca" pod "b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" (UID: "b1e5244c-8ea2-454f-a1fe-ca4333aeba6a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:30.122718 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.122629    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:30.284475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.284436    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:47:30.284722 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.284695    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 31 00:47:30.322107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.322074    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"
Oct 31 00:47:30.372342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.351809    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:30.372342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.351868    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:30.372342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.351962    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:30.372342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.351982    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:30.372342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.352039    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:30.372342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.352058    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:30.394207 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.394109    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:30.394795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.394735    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:30.431672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.430476    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:47:30.627912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.627840    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded"
Oct 31 00:47:30.637292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.637261    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:47:30.638643 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.638526    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:30.639995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.639971    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:47:30.653884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.641837    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:47:30.654510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.654085    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt" event={"ID":"b1e5244c-8ea2-454f-a1fe-ca4333aeba6a","Type":"ContainerDied","Data":"9659e888beb18ebab407997e24208d2413a07c282e6ca5972bfd0c3abc282acb"}
Oct 31 00:47:30.654703 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.654684    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9659e888beb18ebab407997e24208d2413a07c282e6ca5972bfd0c3abc282acb"
Oct 31 00:47:30.654877 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.654864    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:47:30.655022 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.655012    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:30.742212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.742178    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:30.820765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.799137    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:30.820765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.799201    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:30.820765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.799235    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:47:30.820765 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.799330    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:30.890494 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:30.890459    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/health\": context deadline exceeded"
Oct 31 00:47:31.111904 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.111738    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 31 00:47:31.111904 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.111845    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 31 00:47:31.129911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.129774    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 31 00:47:31.129911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.129830    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 31 00:47:31.144812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.144679    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 31 00:47:31.144812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.144738    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 31 00:47:31.186478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.177867    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:47:31.244793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.240396    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 31 00:47:31.244793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.240448    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 31 00:47:31.473827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.463722    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:31.821685 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.821637    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:31.910389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:31.882728    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:32.070807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.000487    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:32.070807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.037945    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:32.207319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.188550    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:32.310771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.310713    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:47:32.311041 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.311022    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:32.311168 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.311155    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:32.311282 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.311271    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:32.311396 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.311384    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:32.542360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.541970    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:32.607167 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.607124    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:32.607862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.607836    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:32.660838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:32.660807    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:33.093640 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:33.093600    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:33.106555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:33.106523    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:33.114960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:33.114907    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:47:33.115197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:33.115165    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:47:33.115784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:33.115763    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:47:33.116128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:33.116108    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:47:36.168248 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.167919    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"]
Oct 31 00:47:36.168948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.168923    4924 cpu_manager.go:414] "containerMap: removing container" podUID="b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" containerName="revoke-keys"
Oct 31 00:47:36.168948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.168947    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" containerName="revoke-keys"
Oct 31 00:47:36.169123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.168984    4924 cpu_manager.go:414] "containerMap: removing container" podUID="54bac04b-501a-4acb-bb7b-6ee4f2f483a6" containerName="revoke-keys"
Oct 31 00:47:36.169123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.168991    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54bac04b-501a-4acb-bb7b-6ee4f2f483a6" containerName="revoke-keys"
Oct 31 00:47:36.169225 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.169182    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" containerName="revoke-keys"
Oct 31 00:47:36.169225 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.169202    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="54bac04b-501a-4acb-bb7b-6ee4f2f483a6" containerName="revoke-keys"
Oct 31 00:47:36.173820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.173724    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.178418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.178357    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:47:36.179288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.179264    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:47:36.181525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.181496    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"]
Oct 31 00:47:36.235237 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.235115    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.236137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.236110    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgbq6\" (UniqueName: \"kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.236465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.236447    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.338089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.338047    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.338294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.338142    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgbq6\" (UniqueName: \"kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.338294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.338277    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.338847 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.338817    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.340994 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.340973    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.346076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.346048    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgbq6\" (UniqueName: \"kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6\") pod \"maas-api-revoke-sub-b7af06a7-5hgmv\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.496421 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.496326    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:36.680085 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.680052    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"]
Oct 31 00:47:36.709643 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.709609    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.711435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.711407    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"]
Oct 31 00:47:36.744639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.744609    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjsn8\" (UniqueName: \"kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.744832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.744665    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.744832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.744691    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.795269 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.795232    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"]
Oct 31 00:47:36.846818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.846780    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjsn8\" (UniqueName: \"kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.846961 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.846836    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.847049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.846960    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.847541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.847523    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.849241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.849222    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:36.854660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:36.854640    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjsn8\" (UniqueName: \"kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8\") pod \"maas-api-revoke-sub-c37a93f0-jxgx2\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:37.040290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:37.040203    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:37.183645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:37.183605    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv" event={"ID":"c8f7a776-684d-4887-8b8b-38d19f5d9fc5","Type":"ContainerStarted","Data":"34d561348b9203b530a7e43bc3e869cb9ec2d6c52a4aef08c1b7caa031accf8d"}
Oct 31 00:47:37.337978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:37.337942    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"]
Oct 31 00:47:37.338316 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:37.338289    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f1d65d_b776_43cd_9c2e_c47ba2c85ab4.slice/crio-d06b6c88140a892286571492a3176e89e1854b6ac3d781b4703d1b4a620f419c WatchSource:0}: Error finding container d06b6c88140a892286571492a3176e89e1854b6ac3d781b4703d1b4a620f419c: Status 404 returned error can't find the container with id d06b6c88140a892286571492a3176e89e1854b6ac3d781b4703d1b4a620f419c
Oct 31 00:47:38.194881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:38.194827    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2" event={"ID":"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4","Type":"ContainerStarted","Data":"d3b7585bb72246133090a2d80a2bff4392b1582727a9bf519270641d2b06642d"}
Oct 31 00:47:38.195601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:38.194899    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2" event={"ID":"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4","Type":"ContainerStarted","Data":"d06b6c88140a892286571492a3176e89e1854b6ac3d781b4703d1b4a620f419c"}
Oct 31 00:47:38.197983 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:38.197940    4924 generic.go:356] "Generic (PLEG): container finished" podID="c8f7a776-684d-4887-8b8b-38d19f5d9fc5" containerID="bec7018d6b4320ddc4ed4afe029a93ff1aa70b07122f8853f892d39b1c667dbe" exitCode=0
Oct 31 00:47:38.197983 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:38.197987    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv" event={"ID":"c8f7a776-684d-4887-8b8b-38d19f5d9fc5","Type":"ContainerDied","Data":"bec7018d6b4320ddc4ed4afe029a93ff1aa70b07122f8853f892d39b1c667dbe"}
Oct 31 00:47:38.215078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:38.214978    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2" podStartSLOduration=2.214963588 podStartE2EDuration="2.214963588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:38.211342911 +0000 UTC m=+858.582126699" watchObservedRunningTime="2026-10-31 00:47:38.214963588 +0000 UTC m=+858.585747375"
Oct 31 00:47:39.211339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.211306    4924 generic.go:356] "Generic (PLEG): container finished" podID="67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" containerID="d3b7585bb72246133090a2d80a2bff4392b1582727a9bf519270641d2b06642d" exitCode=0
Oct 31 00:47:39.211734 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.211389    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2" event={"ID":"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4","Type":"ContainerDied","Data":"d3b7585bb72246133090a2d80a2bff4392b1582727a9bf519270641d2b06642d"}
Oct 31 00:47:39.411436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.411408    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:39.498477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.493134    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6\" (UniqueName: \"kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6\") pod \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") "
Oct 31 00:47:39.498477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.493322    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca\") pod \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") "
Oct 31 00:47:39.498477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.493513    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca\") pod \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\" (UID: \"c8f7a776-684d-4887-8b8b-38d19f5d9fc5\") "
Oct 31 00:47:39.498477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.494419    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca" pod "c8f7a776-684d-4887-8b8b-38d19f5d9fc5" (UID: "c8f7a776-684d-4887-8b8b-38d19f5d9fc5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:39.502986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.502930    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca" pod "c8f7a776-684d-4887-8b8b-38d19f5d9fc5" (UID: "c8f7a776-684d-4887-8b8b-38d19f5d9fc5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:39.505594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.504411    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6" pod "c8f7a776-684d-4887-8b8b-38d19f5d9fc5" (UID: "c8f7a776-684d-4887-8b8b-38d19f5d9fc5"). InnerVolumeSpecName "kube-api-access-lgbq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:39.596372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.596333    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgbq6\" (UniqueName: \"kubernetes.io/projected/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-kube-api-access-lgbq6\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:39.596372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.596366    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:39.596372 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:39.596376    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8f7a776-684d-4887-8b8b-38d19f5d9fc5-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:40.226809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.226673    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"
Oct 31 00:47:40.226809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.226673    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv" event={"ID":"c8f7a776-684d-4887-8b8b-38d19f5d9fc5","Type":"ContainerDied","Data":"34d561348b9203b530a7e43bc3e869cb9ec2d6c52a4aef08c1b7caa031accf8d"}
Oct 31 00:47:40.227354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.226822    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34d561348b9203b530a7e43bc3e869cb9ec2d6c52a4aef08c1b7caa031accf8d"
Oct 31 00:47:40.277140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.277104    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"]
Oct 31 00:47:40.278097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.278068    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c8f7a776-684d-4887-8b8b-38d19f5d9fc5" containerName="revoke-keys"
Oct 31 00:47:40.278097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.278095    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8f7a776-684d-4887-8b8b-38d19f5d9fc5" containerName="revoke-keys"
Oct 31 00:47:40.278318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.278301    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8f7a776-684d-4887-8b8b-38d19f5d9fc5" containerName="revoke-keys"
Oct 31 00:47:40.282491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.282415    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.302119 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.301201    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"]
Oct 31 00:47:40.418684 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.418641    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.418899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.418796    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhnf\" (UniqueName: \"kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.418899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.418824    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.454263 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.454237    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:40.519965 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.519928    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8\" (UniqueName: \"kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8\") pod \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") "
Oct 31 00:47:40.520403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.520378    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca\") pod \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") "
Oct 31 00:47:40.520944 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.520924    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca\") pod \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\" (UID: \"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4\") "
Oct 31 00:47:40.521488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.521461    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.521593 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.521574    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhnf\" (UniqueName: \"kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.521649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.521616    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.522453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.522433    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.524518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.523179    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8" pod "67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" (UID: "67f1d65d-b776-43cd-9c2e-c47ba2c85ab4"). InnerVolumeSpecName "kube-api-access-jjsn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:40.526495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.525884    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.526495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.526181    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca" pod "67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" (UID: "67f1d65d-b776-43cd-9c2e-c47ba2c85ab4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:40.532888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.532863    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca" pod "67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" (UID: "67f1d65d-b776-43cd-9c2e-c47ba2c85ab4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:40.536361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.536340    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhnf\" (UniqueName: \"kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf\") pod \"maas-api-revoke-sub-36ccc54c-ws9wf\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.599876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.599836    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:40.624023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.623912    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:40.624023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.623957    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjsn8\" (UniqueName: \"kubernetes.io/projected/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-kube-api-access-jjsn8\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:40.624023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.623977    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:40.909759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:40.909633    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"]
Oct 31 00:47:41.240214 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.240121    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"
Oct 31 00:47:41.240214 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.240118    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2" event={"ID":"67f1d65d-b776-43cd-9c2e-c47ba2c85ab4","Type":"ContainerDied","Data":"d06b6c88140a892286571492a3176e89e1854b6ac3d781b4703d1b4a620f419c"}
Oct 31 00:47:41.240730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.240261    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d06b6c88140a892286571492a3176e89e1854b6ac3d781b4703d1b4a620f419c"
Oct 31 00:47:41.264694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.264658    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf" event={"ID":"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314","Type":"ContainerStarted","Data":"85ef09db439ffe9026a8f0094cb95870df533b1962eea8850669c5c4370549f7"}
Oct 31 00:47:41.955770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.933798    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"]
Oct 31 00:47:41.975783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.956064    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"]
Oct 31 00:47:41.975783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:41.975148    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"]
Oct 31 00:47:42.298625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:42.296931    4924 generic.go:356] "Generic (PLEG): container finished" podID="826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" containerID="60b19aa8419b3938f72bc4fab753ca6d7c1a4b6fe41175412caf59f5ab1421ff" exitCode=0
Oct 31 00:47:42.298625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:42.297290    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf" event={"ID":"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314","Type":"ContainerDied","Data":"60b19aa8419b3938f72bc4fab753ca6d7c1a4b6fe41175412caf59f5ab1421ff"}
Oct 31 00:47:43.826771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:43.809684    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:44.087508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.086890    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.121769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.098083    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.214782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.213901    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.320386 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.320340    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.327877 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.327829    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.448784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.411869    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.477773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.451276    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:44.503772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.479259    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:47:44.619829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.617864    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.707771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.706814    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:47:44.868912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.866714    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.868912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.866842    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:47:44.886866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.870083    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:44.886866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.878155    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:44.886866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:44.878591    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:47:45.180218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.118089    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:47:45.180218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.118654    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:45.180218 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.180038    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:45.236164 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.236110    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:45.297775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.253644    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:45.352648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.326029    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:45.352648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.326494    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:45.354153 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.353052    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:45.379322 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.379281    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:47:45.379632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.379610    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:45.420121 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.420068    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:45.472956 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.420851    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:45.489790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.480254    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:45.489790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.489234    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:47:45.529773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.526709    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca\") pod \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") "
Oct 31 00:47:45.582772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.552165    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca" pod "826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" (UID: "826c98b9-62fc-4e93-b0ae-6b0ebfcf0314"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:45.582772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.553402    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:45.582772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.576840    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:45.675316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.646737    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:47:45.675316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.660581    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:45.675316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.670545    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf\" (UniqueName: \"kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf\") pod \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") "
Oct 31 00:47:45.675316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.670631    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca\") pod \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\" (UID: \"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314\") "
Oct 31 00:47:45.675316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.671562    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:45.762782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.762426    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.034825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.787902    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Oct 31 00:47:46.075949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.870425    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.111888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.989344    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:46.152773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:45.989487    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.176919 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.029337    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.203770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.146000    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.280167 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.146065    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:47:46.324072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.281519    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca" pod "826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" (UID: "826c98b9-62fc-4e93-b0ae-6b0ebfcf0314"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:46.359216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.359175    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:46.377062 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.356988    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:47:46.537788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.397297    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.537788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.398373    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf" pod "826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" (UID: "826c98b9-62fc-4e93-b0ae-6b0ebfcf0314"). InnerVolumeSpecName "kube-api-access-9xhnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:46.537788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.420512    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:47:46.607884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.599978    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:47:46.607884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.600059    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:46.607884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.600097    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:47:46.607884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.600129    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:46.607884 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.600173    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:47:46.664093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.661911    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.840726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.745374    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.840726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.745484    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:47:46.840726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.767162    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xhnf\" (UniqueName: \"kubernetes.io/projected/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314-kube-api-access-9xhnf\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:46.840726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.816847    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:46.909259 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.908638    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:46.981848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.959269    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"
Oct 31 00:47:47.041016 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:46.977941    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf" event={"ID":"826c98b9-62fc-4e93-b0ae-6b0ebfcf0314","Type":"ContainerDied","Data":"85ef09db439ffe9026a8f0094cb95870df533b1962eea8850669c5c4370549f7"}
Oct 31 00:47:47.092076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.054875    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85ef09db439ffe9026a8f0094cb95870df533b1962eea8850669c5c4370549f7"
Oct 31 00:47:47.092356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.081733    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:47.092841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.082460    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:47.194498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.194382    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:47.278547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.278500    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:47:47.284394 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.284353    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:47:47.286224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.285024    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:47:47.340706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.340644    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:47.383869 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.362520    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:47:47.398163 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.398133    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:47:47.398794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.398772    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:47.399432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.399266    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:47.405347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.404523    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:47:47.428339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.418630    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:47.448347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.430514    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:47:47.451375 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.451345    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:47.511415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.511332    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:47:47.550376 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:47.550343    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"]
Oct 31 00:47:48.451769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.450958    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"]
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.452303    4924 cpu_manager.go:414] "containerMap: removing container" podUID="826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" containerName="revoke-keys"
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.452329    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" containerName="revoke-keys"
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.452353    4924 cpu_manager.go:414] "containerMap: removing container" podUID="67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" containerName="revoke-keys"
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.452361    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" containerName="revoke-keys"
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.452509    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" containerName="revoke-keys"
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.452525    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" containerName="revoke-keys"
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.459394    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"]
Oct 31 00:47:48.462922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.459505    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.480766 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.464918    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:47:48.488515 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.488484    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:47:48.537648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.537615    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4frm\" (UniqueName: \"kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.538265 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.538243    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.538440 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.538429    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.640286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.640246    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4frm\" (UniqueName: \"kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.640465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.640297    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.640465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.640346    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.641089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.641056    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.642909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.642884    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.660208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.660172    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4frm\" (UniqueName: \"kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm\") pod \"maas-api-revoke-sub-dc74f563-gczhp\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.873061 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.873023    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:48.891432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.891400    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"]
Oct 31 00:47:48.904070 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.899254    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:48.904070 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.903790    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"]
Oct 31 00:47:48.945683 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.945646    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrp8v\" (UniqueName: \"kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:48.945896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.945814    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:48.946062 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:48.946046    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.047562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.047525    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrp8v\" (UniqueName: \"kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.047738 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.047582    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.047812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.047782    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.048256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.048237    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.050145 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.050125    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.055594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.055573    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrp8v\" (UniqueName: \"kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v\") pod \"maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.187868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.185546    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"]
Oct 31 00:47:49.242880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.242844    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:49.541402 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:49.541369    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17018251_208e_48f0_a243_091f46783e5a.slice/crio-6ab7acf61e5401dd2565550863ae1ca797b2757356660d0185ac0603434d94c1 WatchSource:0}: Error finding container 6ab7acf61e5401dd2565550863ae1ca797b2757356660d0185ac0603434d94c1: Status 404 returned error can't find the container with id 6ab7acf61e5401dd2565550863ae1ca797b2757356660d0185ac0603434d94c1
Oct 31 00:47:49.542171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.542145    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"]
Oct 31 00:47:49.994559 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.994521    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn" event={"ID":"17018251-208e-48f0-a243-091f46783e5a","Type":"ContainerStarted","Data":"ba2c1b1afc8162f99e9f9a6a75b2e2cea373cb9b6c5180289943b75fe69d2959"}
Oct 31 00:47:49.994782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.994585    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn" event={"ID":"17018251-208e-48f0-a243-091f46783e5a","Type":"ContainerStarted","Data":"6ab7acf61e5401dd2565550863ae1ca797b2757356660d0185ac0603434d94c1"}
Oct 31 00:47:49.998137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.998101    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp" event={"ID":"1297a896-1e24-469f-bc4c-87e1a44495cd","Type":"ContainerStarted","Data":"6866c1e9b88fdc696b0551393ba95fb417cf8a7df5e55481c05f40d8a93abf0a"}
Oct 31 00:47:49.998302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:49.998166    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp" event={"ID":"1297a896-1e24-469f-bc4c-87e1a44495cd","Type":"ContainerStarted","Data":"21088396e28f537adae77f6a4e7d25c6548ccd139757d599a673a5f8e927dd5a"}
Oct 31 00:47:50.022422 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:50.022364    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn" podStartSLOduration=2.02234671 podStartE2EDuration="2.02234671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:50.01257288 +0000 UTC m=+870.383356688" watchObservedRunningTime="2026-10-31 00:47:50.02234671 +0000 UTC m=+870.393130498"
Oct 31 00:47:50.036226 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:50.036161    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp" podStartSLOduration=2.036141746 podStartE2EDuration="2.036141746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:50.028185937 +0000 UTC m=+870.398969726" watchObservedRunningTime="2026-10-31 00:47:50.036141746 +0000 UTC m=+870.406925534"
Oct 31 00:47:51.012559 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:51.012520    4924 generic.go:356] "Generic (PLEG): container finished" podID="1297a896-1e24-469f-bc4c-87e1a44495cd" containerID="6866c1e9b88fdc696b0551393ba95fb417cf8a7df5e55481c05f40d8a93abf0a" exitCode=0
Oct 31 00:47:51.013114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:51.012588    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp" event={"ID":"1297a896-1e24-469f-bc4c-87e1a44495cd","Type":"ContainerDied","Data":"6866c1e9b88fdc696b0551393ba95fb417cf8a7df5e55481c05f40d8a93abf0a"}
Oct 31 00:47:51.015347 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:51.015320    4924 generic.go:356] "Generic (PLEG): container finished" podID="17018251-208e-48f0-a243-091f46783e5a" containerID="ba2c1b1afc8162f99e9f9a6a75b2e2cea373cb9b6c5180289943b75fe69d2959" exitCode=0
Oct 31 00:47:51.015469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:51.015368    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn" event={"ID":"17018251-208e-48f0-a243-091f46783e5a","Type":"ContainerDied","Data":"ba2c1b1afc8162f99e9f9a6a75b2e2cea373cb9b6c5180289943b75fe69d2959"}
Oct 31 00:47:52.273590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.273561    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:52.318888 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.318527    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:52.407917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.407880    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca\") pod \"17018251-208e-48f0-a243-091f46783e5a\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") "
Oct 31 00:47:52.408116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.407988    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca\") pod \"1297a896-1e24-469f-bc4c-87e1a44495cd\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") "
Oct 31 00:47:52.408116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.408029    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca\") pod \"17018251-208e-48f0-a243-091f46783e5a\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") "
Oct 31 00:47:52.408116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.408050    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v\" (UniqueName: \"kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v\") pod \"17018251-208e-48f0-a243-091f46783e5a\" (UID: \"17018251-208e-48f0-a243-091f46783e5a\") "
Oct 31 00:47:52.408273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.408128    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm\" (UniqueName: \"kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm\") pod \"1297a896-1e24-469f-bc4c-87e1a44495cd\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") "
Oct 31 00:47:52.408349 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.408319    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca" pod "17018251-208e-48f0-a243-091f46783e5a" (UID: "17018251-208e-48f0-a243-091f46783e5a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:52.408521 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.408501    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca\") pod \"1297a896-1e24-469f-bc4c-87e1a44495cd\" (UID: \"1297a896-1e24-469f-bc4c-87e1a44495cd\") "
Oct 31 00:47:52.408982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.408952    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca" pod "1297a896-1e24-469f-bc4c-87e1a44495cd" (UID: "1297a896-1e24-469f-bc4c-87e1a44495cd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:52.409574 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.409550    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:52.409659 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.409580    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17018251-208e-48f0-a243-091f46783e5a-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:52.410950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.410925    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm" pod "1297a896-1e24-469f-bc4c-87e1a44495cd" (UID: "1297a896-1e24-469f-bc4c-87e1a44495cd"). InnerVolumeSpecName "kube-api-access-k4frm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:52.411499 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.411473    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca" pod "17018251-208e-48f0-a243-091f46783e5a" (UID: "17018251-208e-48f0-a243-091f46783e5a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:52.411613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.411590    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v" pod "17018251-208e-48f0-a243-091f46783e5a" (UID: "17018251-208e-48f0-a243-091f46783e5a"). InnerVolumeSpecName "kube-api-access-rrp8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:52.411739 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.411721    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca" pod "1297a896-1e24-469f-bc4c-87e1a44495cd" (UID: "1297a896-1e24-469f-bc4c-87e1a44495cd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:52.510870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.510830    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1297a896-1e24-469f-bc4c-87e1a44495cd-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:52.510870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.510863    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17018251-208e-48f0-a243-091f46783e5a-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:52.510870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.510874    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrp8v\" (UniqueName: \"kubernetes.io/projected/17018251-208e-48f0-a243-091f46783e5a-kube-api-access-rrp8v\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:52.511162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.510884    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4frm\" (UniqueName: \"kubernetes.io/projected/1297a896-1e24-469f-bc4c-87e1a44495cd-kube-api-access-k4frm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:52.551788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:52.551675    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerName="payload-processing" containerID="cri-o://5a0df213f4dfa9769e395268de4c653aa91f056ff3336f7a2b435571d81a3d08" gracePeriod=25
Oct 31 00:47:53.037013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.036981    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"
Oct 31 00:47:53.037215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.037004    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn" event={"ID":"17018251-208e-48f0-a243-091f46783e5a","Type":"ContainerDied","Data":"6ab7acf61e5401dd2565550863ae1ca797b2757356660d0185ac0603434d94c1"}
Oct 31 00:47:53.037215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.037065    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ab7acf61e5401dd2565550863ae1ca797b2757356660d0185ac0603434d94c1"
Oct 31 00:47:53.038995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.038971    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"
Oct 31 00:47:53.038995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.038982    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp" event={"ID":"1297a896-1e24-469f-bc4c-87e1a44495cd","Type":"ContainerDied","Data":"21088396e28f537adae77f6a4e7d25c6548ccd139757d599a673a5f8e927dd5a"}
Oct 31 00:47:53.039141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.039033    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21088396e28f537adae77f6a4e7d25c6548ccd139757d599a673a5f8e927dd5a"
Oct 31 00:47:53.609360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.609323    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"]
Oct 31 00:47:53.610087 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.610071    4924 cpu_manager.go:414] "containerMap: removing container" podUID="1297a896-1e24-469f-bc4c-87e1a44495cd" containerName="revoke-keys"
Oct 31 00:47:53.610138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.610088    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1297a896-1e24-469f-bc4c-87e1a44495cd" containerName="revoke-keys"
Oct 31 00:47:53.610138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.610123    4924 cpu_manager.go:414] "containerMap: removing container" podUID="17018251-208e-48f0-a243-091f46783e5a" containerName="revoke-keys"
Oct 31 00:47:53.610138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.610129    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17018251-208e-48f0-a243-091f46783e5a" containerName="revoke-keys"
Oct 31 00:47:53.610241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.610231    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="17018251-208e-48f0-a243-091f46783e5a" containerName="revoke-keys"
Oct 31 00:47:53.610276 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.610247    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="1297a896-1e24-469f-bc4c-87e1a44495cd" containerName="revoke-keys"
Oct 31 00:47:53.614643 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.614619    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.618328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.618302    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:47:53.618613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.618592    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:47:53.622658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.622594    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"]
Oct 31 00:47:53.636930 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.636900    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.639999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.639973    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"]
Oct 31 00:47:53.649337 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.648940    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"]
Oct 31 00:47:53.655436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.655407    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"]
Oct 31 00:47:53.735389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.735344    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.735608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.735407    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.735608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.735554    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.735697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.735612    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btxnv\" (UniqueName: \"kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.735733 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.735692    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.735803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.735723    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9mks\" (UniqueName: \"kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.837312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.837274    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.837312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.837320    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.837568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.837387    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.837568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.837436    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btxnv\" (UniqueName: \"kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.837568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.837488    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.837568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.837518    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9mks\" (UniqueName: \"kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.838506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.838296    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.838506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.838377    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.840020 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.839992    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.840274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.840251    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.863384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.863319    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9mks\" (UniqueName: \"kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks\") pod \"maas-api-revoke-sub-dcfa16ae-vqlwc\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:53.863489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.863413    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btxnv\" (UniqueName: \"kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv\") pod \"maas-api-revoke-sub-852b427e-wn7wl\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.945619 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.945169    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:53.958228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:53.958195    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:54.270282 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:54.270253    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fb9dd6_b8c4_42df_be1b_4f01e70ea0e5.slice/crio-95f01d27d88e882ed753622ffd68c68e6c05938bbbca5cf79ba0bee99a2574e1 WatchSource:0}: Error finding container 95f01d27d88e882ed753622ffd68c68e6c05938bbbca5cf79ba0bee99a2574e1: Status 404 returned error can't find the container with id 95f01d27d88e882ed753622ffd68c68e6c05938bbbca5cf79ba0bee99a2574e1
Oct 31 00:47:54.270803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:54.270779    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"]
Oct 31 00:47:54.295210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:54.295173    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:47:54.302334 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:47:54.302305    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47772251_ea7e_4c8d_a541_0b81d7015d2d.slice/crio-6520567380ef7ea74df173a63ecaff1d900d9cc694302eaac55c30a7e81e3d64 WatchSource:0}: Error finding container 6520567380ef7ea74df173a63ecaff1d900d9cc694302eaac55c30a7e81e3d64: Status 404 returned error can't find the container with id 6520567380ef7ea74df173a63ecaff1d900d9cc694302eaac55c30a7e81e3d64
Oct 31 00:47:54.306279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:54.306241    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"]
Oct 31 00:47:55.066411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:55.066366    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl" event={"ID":"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5","Type":"ContainerStarted","Data":"d4d8253d21a9d9320fa69b5c9b7f1f76598d9b0b3e4de57d4b56524320e7650a"}
Oct 31 00:47:55.066909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:55.066429    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl" event={"ID":"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5","Type":"ContainerStarted","Data":"95f01d27d88e882ed753622ffd68c68e6c05938bbbca5cf79ba0bee99a2574e1"}
Oct 31 00:47:55.068430 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:55.068400    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc" event={"ID":"47772251-ea7e-4c8d-a541-0b81d7015d2d","Type":"ContainerStarted","Data":"f06fc8c0f355ef9f7183a1dbeb8ca8711e7d31703a5fe75f8b8fb51c60f7c6b2"}
Oct 31 00:47:55.068561 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:55.068445    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc" event={"ID":"47772251-ea7e-4c8d-a541-0b81d7015d2d","Type":"ContainerStarted","Data":"6520567380ef7ea74df173a63ecaff1d900d9cc694302eaac55c30a7e81e3d64"}
Oct 31 00:47:55.086292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:55.086244    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl" podStartSLOduration=2.086230946 podStartE2EDuration="2.086230946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:55.082798112 +0000 UTC m=+875.453581896" watchObservedRunningTime="2026-10-31 00:47:55.086230946 +0000 UTC m=+875.457014735"
Oct 31 00:47:55.118660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:55.118588    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc" podStartSLOduration=2.118550807 podStartE2EDuration="2.118550807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:55.117855434 +0000 UTC m=+875.488639236" watchObservedRunningTime="2026-10-31 00:47:55.118550807 +0000 UTC m=+875.489334596"
Oct 31 00:47:56.080309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:56.080273    4924 generic.go:356] "Generic (PLEG): container finished" podID="47772251-ea7e-4c8d-a541-0b81d7015d2d" containerID="f06fc8c0f355ef9f7183a1dbeb8ca8711e7d31703a5fe75f8b8fb51c60f7c6b2" exitCode=0
Oct 31 00:47:56.080784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:56.080349    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc" event={"ID":"47772251-ea7e-4c8d-a541-0b81d7015d2d","Type":"ContainerDied","Data":"f06fc8c0f355ef9f7183a1dbeb8ca8711e7d31703a5fe75f8b8fb51c60f7c6b2"}
Oct 31 00:47:56.082152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:56.082130    4924 generic.go:356] "Generic (PLEG): container finished" podID="d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" containerID="d4d8253d21a9d9320fa69b5c9b7f1f76598d9b0b3e4de57d4b56524320e7650a" exitCode=0
Oct 31 00:47:56.082260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:56.082212    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl" event={"ID":"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5","Type":"ContainerDied","Data":"d4d8253d21a9d9320fa69b5c9b7f1f76598d9b0b3e4de57d4b56524320e7650a"}
Oct 31 00:47:57.005469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:57.005352    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" containerID="cri-o://bb29103e6a8f0307228b7b7bf4e459917216b7a40b7d089bdc59258538278290" gracePeriod=45
Oct 31 00:47:57.050675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:57.047276    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" containerID="cri-o://54e50c60c8fc3d45db270772a102bb7ae30d1898dc93a7d615f8ba7a656f8e3d" gracePeriod=45
Oct 31 00:47:57.164511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:57.155039    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" containerID="cri-o://8b5a1c9ad3f33646336db16a4be7b4f5e77277bd116f64488ac4ce733aea8a42" gracePeriod=45
Oct 31 00:47:57.321194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:57.318267    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": dial tcp 10.132.0.67:8000: connect: connection refused"
Oct 31 00:47:57.678340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:57.672296    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": dial tcp 10.132.0.71:8000: connect: connection refused"
Oct 31 00:47:58.406943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.406278    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:58.464779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.462495    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:58.464779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.462995    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:47:58.507771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.503217    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:58.507771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.504389    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": dial tcp 10.132.0.101:8000: connect: connection refused"
Oct 31 00:47:58.533775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.519732    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": dial tcp 10.132.0.67:8000: connect: connection refused"
Oct 31 00:47:58.598773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.591370    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:47:58.694834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.656106    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"]
Oct 31 00:47:58.694834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.619838    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:58.694834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.676125    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": dial tcp 10.132.0.71:8000: connect: connection refused"
Oct 31 00:47:58.694834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.654795    4924 generic.go:356] "Generic (PLEG): container finished" podID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerID="54e50c60c8fc3d45db270772a102bb7ae30d1898dc93a7d615f8ba7a656f8e3d" exitCode=0
Oct 31 00:47:58.773645 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.773605    4924 generic.go:356] "Generic (PLEG): container finished" podID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerID="bb29103e6a8f0307228b7b7bf4e459917216b7a40b7d089bdc59258538278290" exitCode=0
Oct 31 00:47:58.964130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:58.964023    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.021607 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.021572    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:47:59.021903 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.021873    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"]
Oct 31 00:47:59.030193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.030155    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" event={"ID":"c5755e21-767b-4780-8a8a-093aeaa892ba","Type":"ContainerDied","Data":"54e50c60c8fc3d45db270772a102bb7ae30d1898dc93a7d615f8ba7a656f8e3d"}
Oct 31 00:47:59.030432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.030411    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl" event={"ID":"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5","Type":"ContainerDied","Data":"95f01d27d88e882ed753622ffd68c68e6c05938bbbca5cf79ba0bee99a2574e1"}
Oct 31 00:47:59.030574 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.030559    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95f01d27d88e882ed753622ffd68c68e6c05938bbbca5cf79ba0bee99a2574e1"
Oct 31 00:47:59.030690 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.030674    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" event={"ID":"472a32a4-989d-4dc1-a65a-4a8267ef9283","Type":"ContainerDied","Data":"bb29103e6a8f0307228b7b7bf4e459917216b7a40b7d089bdc59258538278290"}
Oct 31 00:47:59.069993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.069953    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.072465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.072428    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:47:59.122783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.121290    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.148169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.143678    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:47:59.217891 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.217801    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca\") pod \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") "
Oct 31 00:47:59.263480 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.263439    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv\" (UniqueName: \"kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv\") pod \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") "
Oct 31 00:47:59.263721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.263700    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca\") pod \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\" (UID: \"d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5\") "
Oct 31 00:47:59.313109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.264300    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca" pod "d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" (UID: "d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:59.313109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.284412    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca" pod "d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" (UID: "d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373261    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373344    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373458    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373559    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v9kf\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373600    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373762    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373815    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.373850    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.374087    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.374233    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:59.375787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.374251    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:59.407771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.406866    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.408271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.408171    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:47:59.461802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.413946    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": dial tcp 10.132.0.101:8000: connect: connection refused"
Oct 31 00:47:59.461802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.414050    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:47:59.467794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.462827    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.486719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.469340    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:47:59.486719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.474503    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:47:59.486719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.476779    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.563104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.492902    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:47:59.563104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.500030    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:47:59.563104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.562272    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.582580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.574158    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.582580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.575707    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:47:59.741813 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.741724    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.768514    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.768615    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.768669    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.784666    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796176    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796322    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796491    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v9kf\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796545    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796765    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796833    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.797468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.796870    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.799796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.799775    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.980773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.738998    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:59.980773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.800870    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.980773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.804539    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:47:59.980773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.961117    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca\") pod \"47772251-ea7e-4c8d-a541-0b81d7015d2d\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") "
Oct 31 00:47:59.980773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.966815    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca\") pod \"47772251-ea7e-4c8d-a541-0b81d7015d2d\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") "
Oct 31 00:47:59.980773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:47:59.966873    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks\" (UniqueName: \"kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks\") pod \"47772251-ea7e-4c8d-a541-0b81d7015d2d\" (UID: \"47772251-ea7e-4c8d-a541-0b81d7015d2d\") "
Oct 31 00:48:00.152769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.110593    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:48:00.294267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.113177    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded"
Oct 31 00:48:00.294267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.126902    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:00.294267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.128407    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:00.294267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.147384    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca" pod "47772251-ea7e-4c8d-a541-0b81d7015d2d" (UID: "47772251-ea7e-4c8d-a541-0b81d7015d2d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:00.294267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.169465    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:00.294267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.270291    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:00.329855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.320438    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:00.359697 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:00.359658    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:48:00.410502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.401520    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": dial tcp 10.132.0.101:8000: connect: connection refused"
Oct 31 00:48:00.412335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.412204    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:00.437803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.413042    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks" pod "47772251-ea7e-4c8d-a541-0b81d7015d2d" (UID: "47772251-ea7e-4c8d-a541-0b81d7015d2d"). InnerVolumeSpecName "kube-api-access-g9mks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:00.552290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.465973    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:00.709296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.690437    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca" pod "47772251-ea7e-4c8d-a541-0b81d7015d2d" (UID: "47772251-ea7e-4c8d-a541-0b81d7015d2d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:00.712771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.667610    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9mks\" (UniqueName: \"kubernetes.io/projected/47772251-ea7e-4c8d-a541-0b81d7015d2d-kube-api-access-g9mks\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:00.716813 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.710003    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:00.724875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.723739    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:00.803927 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.685824    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:48:00.888292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.879288    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:48:00.888292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.879474    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:48:01.014833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.917092    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd" event={"ID":"472a32a4-989d-4dc1-a65a-4a8267ef9283","Type":"ContainerDied","Data":"78b6243b0c914be05dc5510c2ed1801d16542cf797c42a867663b080633b08dc"}
Oct 31 00:48:01.111769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.110897    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78b6243b0c914be05dc5510c2ed1801d16542cf797c42a867663b080633b08dc"
Oct 31 00:48:01.111769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.760807    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:01.144787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.879175    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:48:01.144787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.760884    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:48:01.428210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.428157    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:48:01.429231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.429188    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 31 00:48:01.454006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:00.760859    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:01.454302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.088435    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv" pod "d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" (UID: "d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5"). InnerVolumeSpecName "kube-api-access-btxnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:01.536770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.512111    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:01.867773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.559816    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.112195 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.663847    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.112507 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.663891    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.112633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.776015    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4" podUID="42092e39-d9c6-47f8-b6dd-b4921bdc955b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:02.112767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.801881    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:02.112880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.871428    4924 generic.go:356] "Generic (PLEG): container finished" podID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerID="8b5a1c9ad3f33646336db16a4be7b4f5e77277bd116f64488ac4ce733aea8a42" exitCode=0
Oct 31 00:48:02.113174 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:01.871511    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.381829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.355502    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btxnv\" (UniqueName: \"kubernetes.io/projected/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5-kube-api-access-btxnv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:02.381829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.355604    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/47772251-ea7e-4c8d-a541-0b81d7015d2d-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:02.471773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.440313    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:02.581811 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.524898    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v9kf\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf\") pod \"e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") " pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:02.728894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.726969    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.728894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.727126    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.728894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.727206    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.767788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.754891    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:48:02.853830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.852136    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:02.877450 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:02.877129    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:03.059592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.045397    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:48:03.059592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.051624    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:48:03.059592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.051695    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:48:03.059592 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.051713    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" event={"ID":"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6","Type":"ContainerDied","Data":"8b5a1c9ad3f33646336db16a4be7b4f5e77277bd116f64488ac4ce733aea8a42"}
Oct 31 00:48:03.091764 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.091711    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"
Oct 31 00:48:03.092175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.092151    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"
Oct 31 00:48:03.600978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.600874    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache\") pod \"472a32a4-989d-4dc1-a65a-4a8267ef9283\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") "
Oct 31 00:48:03.606835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.606797    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir\") pod \"c5755e21-767b-4780-8a8a-093aeaa892ba\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") "
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.620071    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache" pod "472a32a4-989d-4dc1-a65a-4a8267ef9283" (UID: "472a32a4-989d-4dc1-a65a-4a8267ef9283"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.624886    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir" pod "c5755e21-767b-4780-8a8a-093aeaa892ba" (UID: "c5755e21-767b-4780-8a8a-093aeaa892ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.737309    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"]
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738514    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738537    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738556    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" containerName="revoke-keys"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738565    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" containerName="revoke-keys"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738599    4924 cpu_manager.go:414] "containerMap: removing container" podUID="47772251-ea7e-4c8d-a541-0b81d7015d2d" containerName="revoke-keys"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738607    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47772251-ea7e-4c8d-a541-0b81d7015d2d" containerName="revoke-keys"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738643    4924 cpu_manager.go:414] "containerMap: removing container" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738651    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738851    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" containerName="main"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738867    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" containerName="revoke-keys"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738881    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="47772251-ea7e-4c8d-a541-0b81d7015d2d" containerName="revoke-keys"
Oct 31 00:48:03.739555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.738892    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" containerName="main"
Oct 31 00:48:03.873114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.869610    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:03.873114 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.869641    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:03.974650 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.974615    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir\") pod \"472a32a4-989d-4dc1-a65a-4a8267ef9283\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") "
Oct 31 00:48:03.975343 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.975208    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir" pod "472a32a4-989d-4dc1-a65a-4a8267ef9283" (UID: "472a32a4-989d-4dc1-a65a-4a8267ef9283"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:03.975584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.975553    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm\") pod \"c5755e21-767b-4780-8a8a-093aeaa892ba\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") "
Oct 31 00:48:03.976149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.976127    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home\") pod \"c5755e21-767b-4780-8a8a-093aeaa892ba\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") "
Oct 31 00:48:03.976382 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.976365    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs\" (UniqueName: \"kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs\") pod \"c5755e21-767b-4780-8a8a-093aeaa892ba\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") "
Oct 31 00:48:03.976687 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.976667    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache\") pod \"c5755e21-767b-4780-8a8a-093aeaa892ba\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") "
Oct 31 00:48:03.976873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.976854    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr\" (UniqueName: \"kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr\") pod \"472a32a4-989d-4dc1-a65a-4a8267ef9283\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") "
Oct 31 00:48:03.977067 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.977035    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home\") pod \"472a32a4-989d-4dc1-a65a-4a8267ef9283\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") "
Oct 31 00:48:03.978078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.978045    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs\" (UniqueName: \"kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs\") pod \"c5755e21-767b-4780-8a8a-093aeaa892ba\" (UID: \"c5755e21-767b-4780-8a8a-093aeaa892ba\") "
Oct 31 00:48:03.978273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.978240    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs\" (UniqueName: \"kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs\") pod \"472a32a4-989d-4dc1-a65a-4a8267ef9283\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") "
Oct 31 00:48:03.978451 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.978427    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm\") pod \"472a32a4-989d-4dc1-a65a-4a8267ef9283\" (UID: \"472a32a4-989d-4dc1-a65a-4a8267ef9283\") "
Oct 31 00:48:03.980013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.979988    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.002832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.998329    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:04.028770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.012986    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc" event={"ID":"47772251-ea7e-4c8d-a541-0b81d7015d2d","Type":"ContainerDied","Data":"6520567380ef7ea74df173a63ecaff1d900d9cc694302eaac55c30a7e81e3d64"}
Oct 31 00:48:04.028770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.013057    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6520567380ef7ea74df173a63ecaff1d900d9cc694302eaac55c30a7e81e3d64"
Oct 31 00:48:04.028770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.019037    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.036293 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.036265    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:48:04.036823 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.036798    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:48:04.039303 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.039237    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:04.040417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.040359    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.051772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.048823    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.051772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.049080    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.051772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.049248    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.051772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.049396    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.101:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.977413    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home" pod "472a32a4-989d-4dc1-a65a-4a8267ef9283" (UID: "472a32a4-989d-4dc1-a65a-4a8267ef9283"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.977589    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home" pod "c5755e21-767b-4780-8a8a-093aeaa892ba" (UID: "c5755e21-767b-4780-8a8a-093aeaa892ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:03.981024    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache" pod "c5755e21-767b-4780-8a8a-093aeaa892ba" (UID: "c5755e21-767b-4780-8a8a-093aeaa892ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.100325    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.101944    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.103117    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.104549    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.104575    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.104591    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.104738    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:48:04.116290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.114541    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.217498    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.218501    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir\") pod \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") "
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.218577    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs\" (UniqueName: \"kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs\") pod \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") "
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.218615    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh\" (UniqueName: \"kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh\") pod \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") "
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.218774    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache\") pod \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") "
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.218993    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm\") pod \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") "
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.219076    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home\") pod \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\" (UID: \"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6\") "
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.219311    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n4q8\" (UniqueName: \"kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.219571    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.219674    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.238913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.223144    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir" pod "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" (UID: "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.259210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.254933    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home" pod "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" (UID: "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.259502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.259469    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:48:04.259660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.259634    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:04.287861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.281099    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache" pod "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" (UID: "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.395976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.395878    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:48:04.396548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.396508    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.396668 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.396647    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.396885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.396866    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n4q8\" (UniqueName: \"kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.397132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.397116    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.397192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.397139    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.397192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.397154    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.417412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.417379    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.418076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.418054    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.543297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.542844    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:48:04.543297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.543045    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"]
Oct 31 00:48:04.543297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.543104    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:48:04.543297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.543130    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"]
Oct 31 00:48:04.544484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.544230    4924 cpu_manager.go:414] "containerMap: removing container" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main"
Oct 31 00:48:04.544484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.544249    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main"
Oct 31 00:48:04.544669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.544540    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" containerName="main"
Oct 31 00:48:04.573055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.572033    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n4q8\" (UniqueName: \"kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8\") pod \"maas-api-revoke-sub-1a2259cd-x2dfm\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.573055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.572269    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh" pod "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" (UID: "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6"). InnerVolumeSpecName "kube-api-access-zx2nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:04.580724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.577913    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm" pod "472a32a4-989d-4dc1-a65a-4a8267ef9283" (UID: "472a32a4-989d-4dc1-a65a-4a8267ef9283"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.580724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.578080    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr" pod "472a32a4-989d-4dc1-a65a-4a8267ef9283" (UID: "472a32a4-989d-4dc1-a65a-4a8267ef9283"). InnerVolumeSpecName "kube-api-access-tpvxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:04.596341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.593923    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs" pod "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" (UID: "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:04.596341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.594413    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm" pod "c5755e21-767b-4780-8a8a-093aeaa892ba" (UID: "c5755e21-767b-4780-8a8a-093aeaa892ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.596341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.594695    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs" pod "c5755e21-767b-4780-8a8a-093aeaa892ba" (UID: "c5755e21-767b-4780-8a8a-093aeaa892ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:04.604618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.604220    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"]
Oct 31 00:48:04.604618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.604285    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"]
Oct 31 00:48:04.607550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.605254    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.610791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.608204    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm" pod "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" (UID: "1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:04.614410 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.614209    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv" event={"ID":"c5755e21-767b-4780-8a8a-093aeaa892ba","Type":"ContainerDied","Data":"f9eacf33ebd44272e5df1f269a886531c6bf4523bd82a2839b1844d3a6679f23"}
Oct 31 00:48:04.614410 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.614266    4924 scope.go:122] "RemoveContainer" containerID="54e50c60c8fc3d45db270772a102bb7ae30d1898dc93a7d615f8ba7a656f8e3d"
Oct 31 00:48:04.616648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.616627    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"
Oct 31 00:48:04.622508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.622487    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"
Oct 31 00:48:04.631181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.631143    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"
Oct 31 00:48:04.640835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.631477    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4" event={"ID":"1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6","Type":"ContainerDied","Data":"3834ffcd48be88f6b4510e5b07e992ba6bac4452884fc03c78b4b4556b4712a0"}
Oct 31 00:48:04.645801 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.643913    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs" pod "c5755e21-767b-4780-8a8a-093aeaa892ba" (UID: "c5755e21-767b-4780-8a8a-093aeaa892ba"). InnerVolumeSpecName "kube-api-access-pzcqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:04.668929 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.660129    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs" pod "472a32a4-989d-4dc1-a65a-4a8267ef9283" (UID: "472a32a4-989d-4dc1-a65a-4a8267ef9283"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:04.671579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.671439    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:04.680589 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.671369    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/472a32a4-989d-4dc1-a65a-4a8267ef9283-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.687864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.687840    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.688093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.688072    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx2nh\" (UniqueName: \"kubernetes.io/projected/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-kube-api-access-zx2nh\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.688217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.688203    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5755e21-767b-4780-8a8a-093aeaa892ba-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.688310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.688299    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5755e21-767b-4780-8a8a-093aeaa892ba-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.688403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.688389    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpvxr\" (UniqueName: \"kubernetes.io/projected/472a32a4-989d-4dc1-a65a-4a8267ef9283-kube-api-access-tpvxr\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.688495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.688483    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.804878 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.804828    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": context deadline exceeded"
Oct 31 00:48:04.820375 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.820340    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:48:04.822931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.822903    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7k2s\" (UniqueName: \"kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.823309 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.823123    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.823562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.823537    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.823805 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.823787    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzcqs\" (UniqueName: \"kubernetes.io/projected/c5755e21-767b-4780-8a8a-093aeaa892ba-kube-api-access-pzcqs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.823879 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.823812    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472a32a4-989d-4dc1-a65a-4a8267ef9283-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:04.837936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.835432    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:48:04.837936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.836429    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:48:04.839041 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:04.839007    4924 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fb9dd6_b8c4_42df_be1b_4f01e70ea0e5.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47772251_ea7e_4c8d_a541_0b81d7015d2d.slice\": RecentStats: unable to find data in memory cache]"
Oct 31 00:48:04.848864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.848711    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:48:04.858192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.851662    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:48:04.858192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.853057    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:48:04.859656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.859054    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:48:04.860934 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.860447    4924 scope.go:122] "RemoveContainer" containerID="8b5a1c9ad3f33646336db16a4be7b4f5e77277bd116f64488ac4ce733aea8a42"
Oct 31 00:48:04.875632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.870496    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:48:04.890773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.887715    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:48:04.925495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.924958    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7k2s\" (UniqueName: \"kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.925495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.925091    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.925495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.925308    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.926837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.926240    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.931528 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.931474    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:04.942459 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:04.942330    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7k2s\" (UniqueName: \"kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s\") pod \"maas-api-revoke-sub-6b66834d-ms2qk\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:05.018204 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.018168    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:05.052941 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.052878    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:48:05.053034 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.053005    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:48:05.053096 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.053074    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:48:05.078055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.075831    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"]
Oct 31 00:48:05.090510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.090436    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"]
Oct 31 00:48:05.099301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.099228    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/premium-w3-1d4a7e-kserve-7c5bc96bb5-ww4kd"]
Oct 31 00:48:05.146776 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.139897    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"]
Oct 31 00:48:05.162639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.156997    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/distinct-w3-1d4a7e-kserve-696f558b7b-t8wh4"]
Oct 31 00:48:05.218881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.218849    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"]
Oct 31 00:48:05.256823 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.256214    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"]
Oct 31 00:48:05.266848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.266812    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-1d4a7e/simulator-w3-1d4a7e-kserve-7dc97c64-8wbfv"]
Oct 31 00:48:05.358807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.358769    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"]
Oct 31 00:48:05.396491 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:05.396458    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0c9073_46bd_4718_8d47_84a00663c82d.slice/crio-8cdf31b95b628f56bd8ad15116fa8af4b2fbab083d5363e233135be8b9b72f9b WatchSource:0}: Error finding container 8cdf31b95b628f56bd8ad15116fa8af4b2fbab083d5363e233135be8b9b72f9b: Status 404 returned error can't find the container with id 8cdf31b95b628f56bd8ad15116fa8af4b2fbab083d5363e233135be8b9b72f9b
Oct 31 00:48:05.665111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.665062    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" event={"ID":"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d","Type":"ContainerStarted","Data":"dbe36733f7b7424cf1ea8e96ff575daa817529cf3564f8de0e1d221d6bd00c9f"}
Oct 31 00:48:05.665610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.665187    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" event={"ID":"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d","Type":"ContainerStarted","Data":"5598ea3ed2bc72ad40e32a58d24d128ce442877d9ef1b974c5e0d33edf5614fb"}
Oct 31 00:48:05.676365 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.676321    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk" event={"ID":"fb0c9073-46bd-4718-8d47-84a00663c82d","Type":"ContainerStarted","Data":"8cdf31b95b628f56bd8ad15116fa8af4b2fbab083d5363e233135be8b9b72f9b"}
Oct 31 00:48:05.680887 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.680855    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm" event={"ID":"ef3ebc53-8a7d-41c6-8117-45372d57513c","Type":"ContainerStarted","Data":"926d01938d344c29aaee5e9b7f6999e19d656a6e376a5ce7ca94324bbf84d8f3"}
Oct 31 00:48:05.746015 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.745956    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podStartSLOduration=7.745941784 podStartE2EDuration="7.745941784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:05.710961993 +0000 UTC m=+886.081745784" watchObservedRunningTime="2026-10-31 00:48:05.745941784 +0000 UTC m=+886.116725649"
Oct 31 00:48:05.769295 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.769262    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6" path="/var/lib/kubelet/pods/1df650ec-c09d-4e4d-8ebb-6de4cc5ac6c6/volumes"
Oct 31 00:48:05.769982 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.769963    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="472a32a4-989d-4dc1-a65a-4a8267ef9283" path="/var/lib/kubelet/pods/472a32a4-989d-4dc1-a65a-4a8267ef9283/volumes"
Oct 31 00:48:05.770663 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.770640    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5755e21-767b-4780-8a8a-093aeaa892ba" path="/var/lib/kubelet/pods/c5755e21-767b-4780-8a8a-093aeaa892ba/volumes"
Oct 31 00:48:05.882936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:05.882904    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:06.725612 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:06.725556    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk" event={"ID":"fb0c9073-46bd-4718-8d47-84a00663c82d","Type":"ContainerStarted","Data":"7ae2dac79bb566488261765ae5bbfbb8e6151d7b5c4d87780dc6cf3fb011e649"}
Oct 31 00:48:06.727873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:06.727834    4924 generic.go:356] "Generic (PLEG): container finished" podID="ef3ebc53-8a7d-41c6-8117-45372d57513c" containerID="ee28f9b146b3f0acdbc1cf87c20aa1313e96e1780625293608f08909baf265b1" exitCode=0
Oct 31 00:48:06.728005 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:06.727911    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm" event={"ID":"ef3ebc53-8a7d-41c6-8117-45372d57513c","Type":"ContainerDied","Data":"ee28f9b146b3f0acdbc1cf87c20aa1313e96e1780625293608f08909baf265b1"}
Oct 31 00:48:06.742141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:06.741899    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk" podStartSLOduration=7.7418811 podStartE2EDuration="7.7418811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:06.740705184 +0000 UTC m=+887.111488973" watchObservedRunningTime="2026-10-31 00:48:06.7418811 +0000 UTC m=+887.112664891"
Oct 31 00:48:06.879412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:06.879363    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 31 00:48:06.879623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:06.879445    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 31 00:48:07.569410 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.568001    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"]
Oct 31 00:48:07.580001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.579966    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"]
Oct 31 00:48:07.580348 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.580276    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.743997 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.743534    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.743997 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.743900    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.744540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.744004    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff2wp\" (UniqueName: \"kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.745152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.745125    4924 generic.go:356] "Generic (PLEG): container finished" podID="fb0c9073-46bd-4718-8d47-84a00663c82d" containerID="7ae2dac79bb566488261765ae5bbfbb8e6151d7b5c4d87780dc6cf3fb011e649" exitCode=0
Oct 31 00:48:07.745349 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.745314    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk" event={"ID":"fb0c9073-46bd-4718-8d47-84a00663c82d","Type":"ContainerDied","Data":"7ae2dac79bb566488261765ae5bbfbb8e6151d7b5c4d87780dc6cf3fb011e649"}
Oct 31 00:48:07.847584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.847475    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.847584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.847550    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff2wp\" (UniqueName: \"kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.847809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.847781    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.848370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.848343    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.850451 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.850429    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.856756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.855916    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff2wp\" (UniqueName: \"kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp\") pod \"maas-api-revoke-sub-e9a3382d-vvvnk\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:07.880773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.880078    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 31 00:48:07.880773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.880158    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 31 00:48:07.937866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:07.937824    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:08.050117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.049898    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:08.154881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.154779    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca\") pod \"ef3ebc53-8a7d-41c6-8117-45372d57513c\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") "
Oct 31 00:48:08.154881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.154856    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca\") pod \"ef3ebc53-8a7d-41c6-8117-45372d57513c\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") "
Oct 31 00:48:08.155131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.155069    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8\" (UniqueName: \"kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8\") pod \"ef3ebc53-8a7d-41c6-8117-45372d57513c\" (UID: \"ef3ebc53-8a7d-41c6-8117-45372d57513c\") "
Oct 31 00:48:08.155188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.155129    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca" pod "ef3ebc53-8a7d-41c6-8117-45372d57513c" (UID: "ef3ebc53-8a7d-41c6-8117-45372d57513c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:08.155731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.155705    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:08.158785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.158139    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8" pod "ef3ebc53-8a7d-41c6-8117-45372d57513c" (UID: "ef3ebc53-8a7d-41c6-8117-45372d57513c"). InnerVolumeSpecName "kube-api-access-8n4q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:08.158785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.158137    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca" pod "ef3ebc53-8a7d-41c6-8117-45372d57513c" (UID: "ef3ebc53-8a7d-41c6-8117-45372d57513c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:08.257110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.257073    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n4q8\" (UniqueName: \"kubernetes.io/projected/ef3ebc53-8a7d-41c6-8117-45372d57513c-kube-api-access-8n4q8\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:08.257110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.257110    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef3ebc53-8a7d-41c6-8117-45372d57513c-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:08.329992 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:08.329943    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1e0dfe_ec09_45ce_922d_6a21740f3c5b.slice/crio-0c15f394be8eec234ff710a5f4df1c38dbc710f7bcbe0e4ca8749452943e44cd WatchSource:0}: Error finding container 0c15f394be8eec234ff710a5f4df1c38dbc710f7bcbe0e4ca8749452943e44cd: Status 404 returned error can't find the container with id 0c15f394be8eec234ff710a5f4df1c38dbc710f7bcbe0e4ca8749452943e44cd
Oct 31 00:48:08.330488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.330458    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"]
Oct 31 00:48:08.673936 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.673894    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" containerID="cri-o://ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84" gracePeriod=45
Oct 31 00:48:08.758211 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.758169    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk" event={"ID":"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b","Type":"ContainerStarted","Data":"0c15f394be8eec234ff710a5f4df1c38dbc710f7bcbe0e4ca8749452943e44cd"}
Oct 31 00:48:08.759590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.759566    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"
Oct 31 00:48:08.759710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.759597    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm" event={"ID":"ef3ebc53-8a7d-41c6-8117-45372d57513c","Type":"ContainerDied","Data":"926d01938d344c29aaee5e9b7f6999e19d656a6e376a5ce7ca94324bbf84d8f3"}
Oct 31 00:48:08.759710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.759641    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="926d01938d344c29aaee5e9b7f6999e19d656a6e376a5ce7ca94324bbf84d8f3"
Oct 31 00:48:08.759810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.759785    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"]
Oct 31 00:48:08.760635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.760607    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ef3ebc53-8a7d-41c6-8117-45372d57513c" containerName="revoke-keys"
Oct 31 00:48:08.761608 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.761587    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef3ebc53-8a7d-41c6-8117-45372d57513c" containerName="revoke-keys"
Oct 31 00:48:08.761898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.761883    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef3ebc53-8a7d-41c6-8117-45372d57513c" containerName="revoke-keys"
Oct 31 00:48:08.766023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.765961    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.774493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.771581    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 00:48:08.774493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.771639    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-srs6x\""
Oct 31 00:48:08.774493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.771650    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 00:48:08.774493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.771675    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 00:48:08.787872 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.779813    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"]
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.880406    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.880467    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.881927    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.881965    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.882011    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th5x6\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.882041    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.882107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.882074    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.883119 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.882198    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.883119 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.882216    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.946260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.944579    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main" probeResult="failure" output="Get \"https://10.132.0.96:8000/ready\": dial tcp 10.132.0.96:8000: connect: connection refused"
Oct 31 00:48:08.975266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.973579    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"]
Oct 31 00:48:08.984507 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.984253    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.984507 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.984309    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.985018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.984991    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.985123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.985051    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.985182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.985132    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th5x6\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.985236 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.985181    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.985288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.985237    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.986917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.986889    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:08.991094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.991060    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"]
Oct 31 00:48:08.992100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:08.992040    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.028075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.021687    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.028075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.027971    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.056427 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.056359    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.057506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.057450    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.067733 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.067703    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th5x6\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qk8gx\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.154137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.137684    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:09.172609 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.172101    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:09.190003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.189961    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329075    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s\" (UniqueName: \"kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s\") pod \"fb0c9073-46bd-4718-8d47-84a00663c82d\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329128    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home\") pod \"55d7c9b5-8371-493f-a948-842d079ab361\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329236    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs\" (UniqueName: \"kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs\") pod \"55d7c9b5-8371-493f-a948-842d079ab361\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329272    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch\" (UniqueName: \"kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch\") pod \"55d7c9b5-8371-493f-a948-842d079ab361\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329306    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm\") pod \"55d7c9b5-8371-493f-a948-842d079ab361\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329406    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca\") pod \"fb0c9073-46bd-4718-8d47-84a00663c82d\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329539    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir\") pod \"55d7c9b5-8371-493f-a948-842d079ab361\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329575    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache\") pod \"55d7c9b5-8371-493f-a948-842d079ab361\" (UID: \"55d7c9b5-8371-493f-a948-842d079ab361\") "
Oct 31 00:48:09.333770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.329614    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca\") pod \"fb0c9073-46bd-4718-8d47-84a00663c82d\" (UID: \"fb0c9073-46bd-4718-8d47-84a00663c82d\") "
Oct 31 00:48:09.334353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.334112    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home" pod "55d7c9b5-8371-493f-a948-842d079ab361" (UID: "55d7c9b5-8371-493f-a948-842d079ab361"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:09.334691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.334402    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir" pod "55d7c9b5-8371-493f-a948-842d079ab361" (UID: "55d7c9b5-8371-493f-a948-842d079ab361"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:09.334691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.334481    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca" pod "fb0c9073-46bd-4718-8d47-84a00663c82d" (UID: "fb0c9073-46bd-4718-8d47-84a00663c82d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:09.334691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.334612    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache" pod "55d7c9b5-8371-493f-a948-842d079ab361" (UID: "55d7c9b5-8371-493f-a948-842d079ab361"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:09.337620 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.337583    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca" pod "fb0c9073-46bd-4718-8d47-84a00663c82d" (UID: "fb0c9073-46bd-4718-8d47-84a00663c82d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:09.338890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.338853    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs" pod "55d7c9b5-8371-493f-a948-842d079ab361" (UID: "55d7c9b5-8371-493f-a948-842d079ab361"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:09.339098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.339077    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch" pod "55d7c9b5-8371-493f-a948-842d079ab361" (UID: "55d7c9b5-8371-493f-a948-842d079ab361"). InnerVolumeSpecName "kube-api-access-ptpch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:09.339624 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.339587    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s" pod "fb0c9073-46bd-4718-8d47-84a00663c82d" (UID: "fb0c9073-46bd-4718-8d47-84a00663c82d"). InnerVolumeSpecName "kube-api-access-l7k2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:09.340272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.340021    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm" pod "55d7c9b5-8371-493f-a948-842d079ab361" (UID: "55d7c9b5-8371-493f-a948-842d079ab361"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:09.435593 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.435495    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7k2s\" (UniqueName: \"kubernetes.io/projected/fb0c9073-46bd-4718-8d47-84a00663c82d-kube-api-access-l7k2s\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.435849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.435826    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.435977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.435963    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d7c9b5-8371-493f-a948-842d079ab361-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.436085 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.436071    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptpch\" (UniqueName: \"kubernetes.io/projected/55d7c9b5-8371-493f-a948-842d079ab361-kube-api-access-ptpch\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.436213 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.436197    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.436330 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.436316    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.436437 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.436423    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.436536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.436523    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d7c9b5-8371-493f-a948-842d079ab361-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.436637 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.436624    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb0c9073-46bd-4718-8d47-84a00663c82d-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:09.588697 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:09.588640    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1215c9_b847_4ddd_81b0_599213179324.slice/crio-11da07e5b34cbee317621fab823c3731905f0ed8fb27c95226bccf47a0f75c40 WatchSource:0}: Error finding container 11da07e5b34cbee317621fab823c3731905f0ed8fb27c95226bccf47a0f75c40: Status 404 returned error can't find the container with id 11da07e5b34cbee317621fab823c3731905f0ed8fb27c95226bccf47a0f75c40
Oct 31 00:48:09.621130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.603503    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"]
Oct 31 00:48:09.621130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.609875    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerName="payload-processing" containerID="cri-o://0a04fbe3117128c96f38713739570e3490e31a9791d896b12be6eac5a1c67714" gracePeriod=25
Oct 31 00:48:09.839988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.839948    4924 generic.go:356] "Generic (PLEG): container finished" podID="2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" containerID="e9ab42c7d8731b684fb652ee29487468709d785207d51626b0359369e7bc95e6" exitCode=0
Oct 31 00:48:09.840446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.840085    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk" event={"ID":"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b","Type":"ContainerDied","Data":"e9ab42c7d8731b684fb652ee29487468709d785207d51626b0359369e7bc95e6"}
Oct 31 00:48:09.843725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.843695    4924 generic.go:356] "Generic (PLEG): container finished" podID="55d7c9b5-8371-493f-a948-842d079ab361" containerID="ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84" exitCode=0
Oct 31 00:48:09.843911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.843892    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" event={"ID":"55d7c9b5-8371-493f-a948-842d079ab361","Type":"ContainerDied","Data":"ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84"}
Oct 31 00:48:09.844006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.843923    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b" event={"ID":"55d7c9b5-8371-493f-a948-842d079ab361","Type":"ContainerDied","Data":"73e1ec55a5706d50c96eb8c6b31d2d88644110e94e13b148c8b64aaea88408b3"}
Oct 31 00:48:09.844006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.843938    4924 scope.go:122] "RemoveContainer" containerID="ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84"
Oct 31 00:48:09.844122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.844064    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"
Oct 31 00:48:09.858216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.849476    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk" event={"ID":"fb0c9073-46bd-4718-8d47-84a00663c82d","Type":"ContainerDied","Data":"8cdf31b95b628f56bd8ad15116fa8af4b2fbab083d5363e233135be8b9b72f9b"}
Oct 31 00:48:09.858216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.849526    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cdf31b95b628f56bd8ad15116fa8af4b2fbab083d5363e233135be8b9b72f9b"
Oct 31 00:48:09.858216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.849626    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"
Oct 31 00:48:09.858216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.855287    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx" event={"ID":"ef1215c9-b847-4ddd-81b0-599213179324","Type":"ContainerStarted","Data":"11da07e5b34cbee317621fab823c3731905f0ed8fb27c95226bccf47a0f75c40"}
Oct 31 00:48:09.882696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.882662    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 31 00:48:09.882829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.882724    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 31 00:48:09.938379 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.938354    4924 scope.go:122] "RemoveContainer" containerID="ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84"
Oct 31 00:48:09.938817 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:09.938725    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84\": container with ID starting with ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84 not found: ID does not exist" containerID="ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84"
Oct 31 00:48:09.938817 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.938805    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84"} err="failed to get container status \"ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84\": rpc error: code = NotFound desc = could not find container \"ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84\": container with ID starting with ffb8e60e3f0e92607aaab45513e120d5d66a75200beec28452919fab71691d84 not found: ID does not exist"
Oct 31 00:48:09.953770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.953715    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"]
Oct 31 00:48:09.959720 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:09.959686    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-47f370/distinct-2-w6-47f370-kserve-c7c76d4b5-vwk8b"]
Oct 31 00:48:10.878445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:10.878411    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 31 00:48:10.878803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:10.878476    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 31 00:48:11.216831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.207208    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:11.300923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.300882    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca\") pod \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") "
Oct 31 00:48:11.301122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.301102    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp\" (UniqueName: \"kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp\") pod \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") "
Oct 31 00:48:11.301194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.301151    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca\") pod \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\" (UID: \"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b\") "
Oct 31 00:48:11.303545 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.303507    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca" pod "2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" (UID: "2b1e0dfe-ec09-45ce-922d-6a21740f3c5b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:11.335295 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.335239    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca" pod "2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" (UID: "2b1e0dfe-ec09-45ce-922d-6a21740f3c5b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:11.335622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.335586    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp" pod "2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" (UID: "2b1e0dfe-ec09-45ce-922d-6a21740f3c5b"). InnerVolumeSpecName "kube-api-access-ff2wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:11.403613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.403578    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff2wp\" (UniqueName: \"kubernetes.io/projected/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-kube-api-access-ff2wp\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:11.403613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.403611    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:11.403941 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.403627    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:11.707222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.707185    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"]
Oct 31 00:48:11.708376 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708351    4924 cpu_manager.go:414] "containerMap: removing container" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main"
Oct 31 00:48:11.708441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708380    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main"
Oct 31 00:48:11.708441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708400    4924 cpu_manager.go:414] "containerMap: removing container" podUID="2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" containerName="revoke-keys"
Oct 31 00:48:11.708441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708409    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" containerName="revoke-keys"
Oct 31 00:48:11.708441 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708435    4924 cpu_manager.go:414] "containerMap: removing container" podUID="fb0c9073-46bd-4718-8d47-84a00663c82d" containerName="revoke-keys"
Oct 31 00:48:11.708577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708443    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb0c9073-46bd-4718-8d47-84a00663c82d" containerName="revoke-keys"
Oct 31 00:48:11.708639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708625    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="55d7c9b5-8371-493f-a948-842d079ab361" containerName="main"
Oct 31 00:48:11.708670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708643    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" containerName="revoke-keys"
Oct 31 00:48:11.708670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.708657    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb0c9073-46bd-4718-8d47-84a00663c82d" containerName="revoke-keys"
Oct 31 00:48:11.713486 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.713464    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.744115 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.744084    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"]
Oct 31 00:48:11.776192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.776150    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55d7c9b5-8371-493f-a948-842d079ab361" path="/var/lib/kubelet/pods/55d7c9b5-8371-493f-a948-842d079ab361/volumes"
Oct 31 00:48:11.777027 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.777002    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"]
Oct 31 00:48:11.785696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.785657    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.797577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.797538    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"]
Oct 31 00:48:11.821581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.821530    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.822023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.821975    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.822132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.822119    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.822266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.822231    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbbv\" (UniqueName: \"kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.822497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.822480    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bfcj\" (UniqueName: \"kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.822570 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.822559    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.878256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.878216    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 31 00:48:11.878437 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.878290    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 31 00:48:11.881571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.881535    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk" event={"ID":"2b1e0dfe-ec09-45ce-922d-6a21740f3c5b","Type":"ContainerDied","Data":"0c15f394be8eec234ff710a5f4df1c38dbc710f7bcbe0e4ca8749452943e44cd"}
Oct 31 00:48:11.881984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.881597    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c15f394be8eec234ff710a5f4df1c38dbc710f7bcbe0e4ca8749452943e44cd"
Oct 31 00:48:11.882109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.882085    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"
Oct 31 00:48:11.924082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.924036    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bfcj\" (UniqueName: \"kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.924082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.924077    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.924317 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.924192    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.924394 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.924373    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.924465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.924411    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.924465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.924457    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbbv\" (UniqueName: \"kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.925042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.925016    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.925203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.925180    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.927390 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.927357    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:11.927873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.927848    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.933733 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.933710    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbbv\" (UniqueName: \"kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv\") pod \"maas-api-revoke-sub-d346c63e-76rpw\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:11.933967 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:11.933944    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bfcj\" (UniqueName: \"kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj\") pod \"maas-api-revoke-sub-27e19fbd-98wls\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:12.028892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.028860    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:12.113599 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.113543    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:12.413942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.413902    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"]
Oct 31 00:48:12.443144 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:12.443107    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeea9704_4418_474f_afa3_16a00f63233c.slice/crio-f443ec16520d43341cb6f46922ec9b5871f3322867e9599d59fa6b0a1c69406e WatchSource:0}: Error finding container f443ec16520d43341cb6f46922ec9b5871f3322867e9599d59fa6b0a1c69406e: Status 404 returned error can't find the container with id f443ec16520d43341cb6f46922ec9b5871f3322867e9599d59fa6b0a1c69406e
Oct 31 00:48:12.566721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.566262    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"]
Oct 31 00:48:12.988997 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.988948    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw" event={"ID":"feea9704-4418-474f-afa3-16a00f63233c","Type":"ContainerStarted","Data":"f443ec16520d43341cb6f46922ec9b5871f3322867e9599d59fa6b0a1c69406e"}
Oct 31 00:48:13.007893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.992346    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" event={"ID":"32703c40-6595-4a66-acf4-e32a468b2c8b","Type":"ContainerDied","Data":"5a0df213f4dfa9769e395268de4c653aa91f056ff3336f7a2b435571d81a3d08"}
Oct 31 00:48:13.007893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.992121    4924 generic.go:356] "Generic (PLEG): container finished" podID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerID="5a0df213f4dfa9769e395268de4c653aa91f056ff3336f7a2b435571d81a3d08" exitCode=0
Oct 31 00:48:13.007893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:12.999539    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"]
Oct 31 00:48:13.012671 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.012489    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls" event={"ID":"758a34a0-6677-4b2a-ae80-0d9171133763","Type":"ContainerStarted","Data":"70d012af06617bb9b5198e8ddb13551d9021097930ba060e82ed4fc0b5e2ab6a"}
Oct 31 00:48:13.084570 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.084525    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-47f370-cw6wn"]
Oct 31 00:48:13.256099 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.255788    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:13.258639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.258577    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:13.266962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.266929    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:48:13.412674 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.412621    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:48:13.595315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.595279    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume\") pod \"32703c40-6595-4a66-acf4-e32a468b2c8b\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") "
Oct 31 00:48:13.595582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.595561    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l\" (UniqueName: \"kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l\") pod \"32703c40-6595-4a66-acf4-e32a468b2c8b\" (UID: \"32703c40-6595-4a66-acf4-e32a468b2c8b\") "
Oct 31 00:48:13.595733 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.595701    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume" pod "32703c40-6595-4a66-acf4-e32a468b2c8b" (UID: "32703c40-6595-4a66-acf4-e32a468b2c8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:13.596385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.596247    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32703c40-6595-4a66-acf4-e32a468b2c8b-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:13.598967 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.598940    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l" pod "32703c40-6595-4a66-acf4-e32a468b2c8b" (UID: "32703c40-6595-4a66-acf4-e32a468b2c8b"). InnerVolumeSpecName "kube-api-access-4zr5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:13.697971 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.697930    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zr5l\" (UniqueName: \"kubernetes.io/projected/32703c40-6595-4a66-acf4-e32a468b2c8b-kube-api-access-4zr5l\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:13.766056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:13.766017    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17018251-208e-48f0-a243-091f46783e5a" path="/var/lib/kubelet/pods/17018251-208e-48f0-a243-091f46783e5a/volumes"
Oct 31 00:48:14.038196 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.038154    4924 generic.go:356] "Generic (PLEG): container finished" podID="758a34a0-6677-4b2a-ae80-0d9171133763" containerID="e822aa4c17c6bf62308fbb73e008c3cfad5748b47e0845f3552613a5334c92ff" exitCode=0
Oct 31 00:48:14.038655 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.038238    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls" event={"ID":"758a34a0-6677-4b2a-ae80-0d9171133763","Type":"ContainerDied","Data":"e822aa4c17c6bf62308fbb73e008c3cfad5748b47e0845f3552613a5334c92ff"}
Oct 31 00:48:14.040656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.040626    4924 generic.go:356] "Generic (PLEG): container finished" podID="feea9704-4418-474f-afa3-16a00f63233c" containerID="dc9fb93318b9a720ad8e1533505430a506a8357060b9591b1e3a57a1d37dbaa4" exitCode=0
Oct 31 00:48:14.040818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.040698    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw" event={"ID":"feea9704-4418-474f-afa3-16a00f63233c","Type":"ContainerDied","Data":"dc9fb93318b9a720ad8e1533505430a506a8357060b9591b1e3a57a1d37dbaa4"}
Oct 31 00:48:14.042664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.042642    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"
Oct 31 00:48:14.042804 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.042657    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb" event={"ID":"32703c40-6595-4a66-acf4-e32a468b2c8b","Type":"ContainerDied","Data":"11bd3902e3aa776aea5338a802ed23be977f733c69014b485e73862b6c988847"}
Oct 31 00:48:14.042804 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.042718    4924 scope.go:122] "RemoveContainer" containerID="5a0df213f4dfa9769e395268de4c653aa91f056ff3336f7a2b435571d81a3d08"
Oct 31 00:48:14.160704 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.160668    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"]
Oct 31 00:48:14.166984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.166952    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-1d4a7e-55bf976d9c-5nrlb"]
Oct 31 00:48:14.288365 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:14.288327    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"]
Oct 31 00:48:15.280163 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.280136    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:15.325600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.325529    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:15.362894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.362841    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca\") pod \"feea9704-4418-474f-afa3-16a00f63233c\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") "
Oct 31 00:48:15.363082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.362904    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca\") pod \"feea9704-4418-474f-afa3-16a00f63233c\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") "
Oct 31 00:48:15.363082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.363043    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv\" (UniqueName: \"kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv\") pod \"feea9704-4418-474f-afa3-16a00f63233c\" (UID: \"feea9704-4418-474f-afa3-16a00f63233c\") "
Oct 31 00:48:15.363210 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.363175    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj\" (UniqueName: \"kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj\") pod \"758a34a0-6677-4b2a-ae80-0d9171133763\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") "
Oct 31 00:48:15.363276 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.363245    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca\") pod \"758a34a0-6677-4b2a-ae80-0d9171133763\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") "
Oct 31 00:48:15.363828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.363372    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca\") pod \"758a34a0-6677-4b2a-ae80-0d9171133763\" (UID: \"758a34a0-6677-4b2a-ae80-0d9171133763\") "
Oct 31 00:48:15.363828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.363775    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca" pod "feea9704-4418-474f-afa3-16a00f63233c" (UID: "feea9704-4418-474f-afa3-16a00f63233c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:15.364135 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.364102    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca" pod "758a34a0-6677-4b2a-ae80-0d9171133763" (UID: "758a34a0-6677-4b2a-ae80-0d9171133763"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:15.364260 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.364238    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feea9704-4418-474f-afa3-16a00f63233c-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:15.368295 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.368265    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj" pod "758a34a0-6677-4b2a-ae80-0d9171133763" (UID: "758a34a0-6677-4b2a-ae80-0d9171133763"). InnerVolumeSpecName "kube-api-access-2bfcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:15.369142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.369109    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca" pod "758a34a0-6677-4b2a-ae80-0d9171133763" (UID: "758a34a0-6677-4b2a-ae80-0d9171133763"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:15.369258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.369139    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv" pod "feea9704-4418-474f-afa3-16a00f63233c" (UID: "feea9704-4418-474f-afa3-16a00f63233c"). InnerVolumeSpecName "kube-api-access-bsbbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:15.371056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.371028    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca" pod "feea9704-4418-474f-afa3-16a00f63233c" (UID: "feea9704-4418-474f-afa3-16a00f63233c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:15.466314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.466271    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:15.466314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.466309    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feea9704-4418-474f-afa3-16a00f63233c-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:15.466314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.466319    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsbbv\" (UniqueName: \"kubernetes.io/projected/feea9704-4418-474f-afa3-16a00f63233c-kube-api-access-bsbbv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:15.466555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.466328    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bfcj\" (UniqueName: \"kubernetes.io/projected/758a34a0-6677-4b2a-ae80-0d9171133763-kube-api-access-2bfcj\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:15.466555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.466338    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/758a34a0-6677-4b2a-ae80-0d9171133763-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:15.771692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:15.770856    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" path="/var/lib/kubelet/pods/32703c40-6595-4a66-acf4-e32a468b2c8b/volumes"
Oct 31 00:48:16.069275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.069187    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"
Oct 31 00:48:16.069275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.069207    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls" event={"ID":"758a34a0-6677-4b2a-ae80-0d9171133763","Type":"ContainerDied","Data":"70d012af06617bb9b5198e8ddb13551d9021097930ba060e82ed4fc0b5e2ab6a"}
Oct 31 00:48:16.069275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.069262    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70d012af06617bb9b5198e8ddb13551d9021097930ba060e82ed4fc0b5e2ab6a"
Oct 31 00:48:16.070899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.070878    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"
Oct 31 00:48:16.071037 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.070925    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw" event={"ID":"feea9704-4418-474f-afa3-16a00f63233c","Type":"ContainerDied","Data":"f443ec16520d43341cb6f46922ec9b5871f3322867e9599d59fa6b0a1c69406e"}
Oct 31 00:48:16.071037 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.070980    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f443ec16520d43341cb6f46922ec9b5871f3322867e9599d59fa6b0a1c69406e"
Oct 31 00:48:16.268215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268182    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"]
Oct 31 00:48:16.268892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268876    4924 cpu_manager.go:414] "containerMap: removing container" podUID="758a34a0-6677-4b2a-ae80-0d9171133763" containerName="revoke-keys"
Oct 31 00:48:16.268949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268894    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="758a34a0-6677-4b2a-ae80-0d9171133763" containerName="revoke-keys"
Oct 31 00:48:16.268949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268921    4924 cpu_manager.go:414] "containerMap: removing container" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerName="payload-processing"
Oct 31 00:48:16.268949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268927    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerName="payload-processing"
Oct 31 00:48:16.268949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268937    4924 cpu_manager.go:414] "containerMap: removing container" podUID="feea9704-4418-474f-afa3-16a00f63233c" containerName="revoke-keys"
Oct 31 00:48:16.269072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.268941    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="feea9704-4418-474f-afa3-16a00f63233c" containerName="revoke-keys"
Oct 31 00:48:16.269104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.269097    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="32703c40-6595-4a66-acf4-e32a468b2c8b" containerName="payload-processing"
Oct 31 00:48:16.269140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.269109    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="feea9704-4418-474f-afa3-16a00f63233c" containerName="revoke-keys"
Oct 31 00:48:16.269140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.269124    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="758a34a0-6677-4b2a-ae80-0d9171133763" containerName="revoke-keys"
Oct 31 00:48:16.274204 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.274173    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.279633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.279609    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:48:16.279795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.279642    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:48:16.290536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.290472    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"]
Oct 31 00:48:16.322169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.304690    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"]
Oct 31 00:48:16.322169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.313040    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.322169 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.317830    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"]
Oct 31 00:48:16.388985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.388941    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q94l2\" (UniqueName: \"kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.389176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.389049    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.389176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.389113    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.389304 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.389213    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7txsr\" (UniqueName: \"kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.389361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.389339    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.389459 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.389441    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.490938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.490896    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q94l2\" (UniqueName: \"kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.491111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.490960    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.491111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.490983    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.491111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.491061    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7txsr\" (UniqueName: \"kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.491111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.491102    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.491311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.491151    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.491935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.491909    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.492048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.491908    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.493835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.493805    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.494110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.494085    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.499562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.499376    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q94l2\" (UniqueName: \"kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2\") pod \"maas-api-revoke-sub-c960dd27-2w6zg\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.503108 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.503078    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7txsr\" (UniqueName: \"kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr\") pod \"maas-api-revoke-sub-486c3564-p2kbv\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.649825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.648342    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:16.649825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:16.649023    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:16.999976 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:16.999929    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230f3f79_4978_42c9_89a9_f32eaa09257e.slice/crio-5f7dd335e910559462e567dbb0adfec856e38c4498e6730e9eff48e0ccde3917 WatchSource:0}: Error finding container 5f7dd335e910559462e567dbb0adfec856e38c4498e6730e9eff48e0ccde3917: Status 404 returned error can't find the container with id 5f7dd335e910559462e567dbb0adfec856e38c4498e6730e9eff48e0ccde3917
Oct 31 00:48:17.004313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:17.003019    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"]
Oct 31 00:48:17.012251 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:17.012218    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"]
Oct 31 00:48:17.023340 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:17.023306    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod461484c0_90e3_4f1a_959a_847f1e9e935b.slice/crio-c8daa913482376096ee56f040c5f183c5a9aca90f0063065b3e4b81d24180e57 WatchSource:0}: Error finding container c8daa913482376096ee56f040c5f183c5a9aca90f0063065b3e4b81d24180e57: Status 404 returned error can't find the container with id c8daa913482376096ee56f040c5f183c5a9aca90f0063065b3e4b81d24180e57
Oct 31 00:48:17.083508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:17.083464    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv" event={"ID":"461484c0-90e3-4f1a-959a-847f1e9e935b","Type":"ContainerStarted","Data":"c8daa913482376096ee56f040c5f183c5a9aca90f0063065b3e4b81d24180e57"}
Oct 31 00:48:17.084984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:17.084957    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg" event={"ID":"230f3f79-4978-42c9-89a9-f32eaa09257e","Type":"ContainerStarted","Data":"5f7dd335e910559462e567dbb0adfec856e38c4498e6730e9eff48e0ccde3917"}
Oct 31 00:48:18.265080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.263673    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv" event={"ID":"461484c0-90e3-4f1a-959a-847f1e9e935b","Type":"ContainerStarted","Data":"a8d0601f4993f3e0b41e67db2c88212b75e825da7c742cfe390eb1d0794a8f3d"}
Oct 31 00:48:18.366908 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.365434    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg" event={"ID":"230f3f79-4978-42c9-89a9-f32eaa09257e","Type":"ContainerStarted","Data":"c9688770585fee0460b3701db84992379ef384c83b9407ac4204408652ce8f30"}
Oct 31 00:48:18.520773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.518096    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv" podStartSLOduration=2.518072089 podStartE2EDuration="2.518072089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:18.303637307 +0000 UTC m=+898.674421107" watchObservedRunningTime="2026-10-31 00:48:18.518072089 +0000 UTC m=+898.888855870"
Oct 31 00:48:18.605088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.605012    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg" podStartSLOduration=2.604987569 podStartE2EDuration="2.604987569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:18.387259291 +0000 UTC m=+898.758043090" watchObservedRunningTime="2026-10-31 00:48:18.604987569 +0000 UTC m=+898.975771359"
Oct 31 00:48:18.802044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.775359    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:18.885716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.884921    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:18.935545 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:18.934903    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:19.107231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:19.107129    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:19.127824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:19.127774    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:19.162387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:19.162344    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:19.290767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:19.289813    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerName="payload-pre-processing" containerID="cri-o://bb30d968d5bd5e02cbe1d3f6c6ce96a33498f77f26816a371347b06a97497487" gracePeriod=25
Oct 31 00:48:19.347799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:19.320875    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:48:19.562774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:19.562539    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:20.077772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:20.063962    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:48:20.177803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:20.148990    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:20.177803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:20.149857    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:48:20.299849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:20.214803    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:20.302371 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:20.302322    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:20.976262 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:20.976218    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.175329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.167889    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.175329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.170490    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.244876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.244779    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.406052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.373908    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:21.447374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.447310    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.471670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.464122    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:21.673223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.476970    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.673223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.527792    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:48:21.673223 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.581441    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk" podUID="af9d788e-efd8-48d9-94db-cc87b84bb703" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:21.731250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.687444    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6qm4" podUID="42092e39-d9c6-47f8-b6dd-b4921bdc955b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:21.731250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.688249    4924 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-f6w7r container/console-operator namespace/openshift-console-operator: Liveness probe status=failure output="Get \"https://10.132.0.14:8443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:48:21.731250 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.688325    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-console-operator/console-operator-7cd56b8dbc-f6w7r" podUID="041328ee-8f09-4927-85e3-f3cceefe3a53" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:21.980498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:21.980297    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.017731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.017655    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:48:22.169942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.169888    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.178433 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.178381    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.381826 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.381765    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.512230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.483241    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.512230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.484677    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:48:22.512230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.484727    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.532978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.532927    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.616774 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:22.592172    4924 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.442s"
Oct 31 00:48:22.642098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.642006    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"]
Oct 31 00:48:22.938203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.656687    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk" podUID="af9d788e-efd8-48d9-94db-cc87b84bb703" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:22.965490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.656719    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:48:22.965490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.669200    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:48:22.965490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.713865    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:22.965490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.843393    4924 generic.go:356] "Generic (PLEG): container finished" podID="461484c0-90e3-4f1a-959a-847f1e9e935b" containerID="a8d0601f4993f3e0b41e67db2c88212b75e825da7c742cfe390eb1d0794a8f3d" exitCode=0
Oct 31 00:48:22.965490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:22.958221    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" containerID="cri-o://2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a" gracePeriod=30
Oct 31 00:48:23.042371 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.041924    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:23.222000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.221793    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:48:23.341782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.315851    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 31 00:48:23.370804 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.368975    4924 generic.go:356] "Generic (PLEG): container finished" podID="230f3f79-4978-42c9-89a9-f32eaa09257e" containerID="c9688770585fee0460b3701db84992379ef384c83b9407ac4204408652ce8f30" exitCode=0
Oct 31 00:48:23.451786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.443264    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:23.612818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.589309    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 31 00:48:23.721829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.656120    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:48:23.840967 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.840924    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:48:23.841219 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.841190    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:23.947264 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:23.947164    4924 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.029s"
Oct 31 00:48:23.947619 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.947596    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:48:23.967777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.947789    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv" event={"ID":"461484c0-90e3-4f1a-959a-847f1e9e935b","Type":"ContainerDied","Data":"a8d0601f4993f3e0b41e67db2c88212b75e825da7c742cfe390eb1d0794a8f3d"}
Oct 31 00:48:24.012891 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.012845    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"]
Oct 31 00:48:24.013176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.013149    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg" event={"ID":"230f3f79-4978-42c9-89a9-f32eaa09257e","Type":"ContainerDied","Data":"c9688770585fee0460b3701db84992379ef384c83b9407ac4204408652ce8f30"}
Oct 31 00:48:24.013319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.013301    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"]
Oct 31 00:48:24.013485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.013470    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:48:24.037771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.013618    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:48:24.038222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.038189    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:48:24.069772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.020711    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" containerID="cri-o://9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f" gracePeriod=30
Oct 31 00:48:24.069772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.955305    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 31 00:48:24.069772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.955340    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 31 00:48:24.070497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.021388    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" containerID="cri-o://90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15" gracePeriod=30
Oct 31 00:48:24.070571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:23.955274    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:24.071071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.071035    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:24.110625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.110554    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/health\": context deadline exceeded"
Oct 31 00:48:24.140768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.131252    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" containerID="cri-o://963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b" gracePeriod=45
Oct 31 00:48:24.226787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.226664    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": read tcp 10.132.0.2:51414->10.132.0.93:8000: read: connection reset by peer"
Oct 31 00:48:24.251821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.228775    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": dial tcp 10.132.0.93:8000: connect: connection refused"
Oct 31 00:48:24.380782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.362031    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" containerID="cri-o://8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a" gracePeriod=45
Oct 31 00:48:24.430470 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.430425    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" probeResult="failure" output="Get \"https://10.132.0.111:8000/ready\": context deadline exceeded"
Oct 31 00:48:24.562769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.534012    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": read tcp 10.132.0.2:38494->10.132.0.69:8000: read: connection reset by peer"
Oct 31 00:48:24.581782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.572598    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/ready\": context deadline exceeded"
Oct 31 00:48:24.659922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.659873    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:24.808827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.757084    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:24.808827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.757153    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.756835    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/ready\": context deadline exceeded"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.818641    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.850814    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.850885    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.850921    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.850954    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.850997    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.851048    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk"
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.851073    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"]
Oct 31 00:48:24.857611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.851275    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy" containerID="cri-o://56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42" gracePeriod=30
Oct 31 00:48:24.881622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:24.881578    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": dial tcp 10.132.0.69:8000: connect: connection refused"
Oct 31 00:48:25.026305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.026244    4924 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" probeResult="failure" output="Get \"https://10.132.0.72:8000/health\": context deadline exceeded"
Oct 31 00:48:25.053895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.053852    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:25.071471 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.071420    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t" podUID="581a4f76-b7c1-4293-9b06-1079af9b9e3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 31 00:48:25.073465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.073420    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": context deadline exceeded"
Oct 31 00:48:25.195175 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.188649    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:48:25.411804 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.411741    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": dial tcp 10.132.0.93:8000: connect: connection refused"
Oct 31 00:48:25.511152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.509981    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:48:25.513895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.511820    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:48:25.547294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.547262    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:48:25.547834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.547813    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t"
Oct 31 00:48:25.574085 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.560028    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:25.574085 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.560100    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:25.574085 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.562576    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:48:25.584633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.583965    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:48:25.735167 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.735036    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"]
Oct 31 00:48:25.743569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.740809    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"]
Oct 31 00:48:25.890878 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.889350    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": dial tcp 10.132.0.69:8000: connect: connection refused"
Oct 31 00:48:25.913890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.913831    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"]
Oct 31 00:48:25.932767 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:25.932701    4924 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17009d78_c2f2_496f_b820_2b4765f7b5f1.slice/crio-conmon-8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf72a3bf_1d16_4cd3_a5a1_74ab07443025.slice/crio-963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17009d78_c2f2_496f_b820_2b4765f7b5f1.slice/crio-8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf72a3bf_1d16_4cd3_a5a1_74ab07443025.slice/crio-conmon-963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 00:48:25.994799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:25.994698    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:48:26.047029 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.044143    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:48:26.056715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.056678    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:48:26.065003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.063601    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:48:26.318817 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.312653    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:48:26.332793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.331094    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:48:26.382254 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.381573    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"]
Oct 31 00:48:26.383017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.382981    4924 cpu_manager.go:414] "containerMap: removing container" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main"
Oct 31 00:48:26.383286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.383256    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main"
Oct 31 00:48:26.395142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.390911    4924 cpu_manager.go:414] "containerMap: removing container" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main"
Oct 31 00:48:26.395142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.390934    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main"
Oct 31 00:48:26.395142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.391370    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main"
Oct 31 00:48:26.395142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.391398    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerName="main"
Oct 31 00:48:26.398126 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.398102    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.417085    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"]
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.417681    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm\") pod \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.417723    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs\" (UniqueName: \"kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs\") pod \"17009d78-c2f2-496f-b820-2b4765f7b5f1\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.417843    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home\") pod \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418300    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255\" (UniqueName: \"kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255\") pod \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418386    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home\") pod \"17009d78-c2f2-496f-b820-2b4765f7b5f1\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418487    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir\") pod \"17009d78-c2f2-496f-b820-2b4765f7b5f1\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418523    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm\") pod \"17009d78-c2f2-496f-b820-2b4765f7b5f1\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418643    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir\") pod \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418724    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache\") pod \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418850    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs\" (UniqueName: \"kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs\") pod \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\" (UID: \"af72a3bf-1d16-4cd3-a5a1-74ab07443025\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.418942    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt\" (UniqueName: \"kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt\") pod \"17009d78-c2f2-496f-b820-2b4765f7b5f1\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.419016    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache\") pod \"17009d78-c2f2-496f-b820-2b4765f7b5f1\" (UID: \"17009d78-c2f2-496f-b820-2b4765f7b5f1\") "
Oct 31 00:48:26.422809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.420038    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache" pod "17009d78-c2f2-496f-b820-2b4765f7b5f1" (UID: "17009d78-c2f2-496f-b820-2b4765f7b5f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.433502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.428459    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir" pod "af72a3bf-1d16-4cd3-a5a1-74ab07443025" (UID: "af72a3bf-1d16-4cd3-a5a1-74ab07443025"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.433502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.428674    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home" pod "17009d78-c2f2-496f-b820-2b4765f7b5f1" (UID: "17009d78-c2f2-496f-b820-2b4765f7b5f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.433502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.431466    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home" pod "af72a3bf-1d16-4cd3-a5a1-74ab07443025" (UID: "af72a3bf-1d16-4cd3-a5a1-74ab07443025"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.433502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.431695    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir" pod "17009d78-c2f2-496f-b820-2b4765f7b5f1" (UID: "17009d78-c2f2-496f-b820-2b4765f7b5f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.433502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.432423    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs" pod "17009d78-c2f2-496f-b820-2b4765f7b5f1" (UID: "17009d78-c2f2-496f-b820-2b4765f7b5f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:26.434008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.433544    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm" pod "17009d78-c2f2-496f-b820-2b4765f7b5f1" (UID: "17009d78-c2f2-496f-b820-2b4765f7b5f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.434392 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.434340    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache" pod "af72a3bf-1d16-4cd3-a5a1-74ab07443025" (UID: "af72a3bf-1d16-4cd3-a5a1-74ab07443025"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.438326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.438293    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt" pod "17009d78-c2f2-496f-b820-2b4765f7b5f1" (UID: "17009d78-c2f2-496f-b820-2b4765f7b5f1"). InnerVolumeSpecName "kube-api-access-4qtqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:26.449485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.449439    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm" pod "af72a3bf-1d16-4cd3-a5a1-74ab07443025" (UID: "af72a3bf-1d16-4cd3-a5a1-74ab07443025"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:26.450189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.450152    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs" pod "af72a3bf-1d16-4cd3-a5a1-74ab07443025" (UID: "af72a3bf-1d16-4cd3-a5a1-74ab07443025"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:26.450384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.450324    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255" pod "af72a3bf-1d16-4cd3-a5a1-74ab07443025" (UID: "af72a3bf-1d16-4cd3-a5a1-74ab07443025"). InnerVolumeSpecName "kube-api-access-n6255". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:26.456193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.455462    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:26.461503 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.460849    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.529341    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca\") pod \"461484c0-90e3-4f1a-959a-847f1e9e935b\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") "
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.529467    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca\") pod \"461484c0-90e3-4f1a-959a-847f1e9e935b\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") "
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.529609    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr\" (UniqueName: \"kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr\") pod \"461484c0-90e3-4f1a-959a-847f1e9e935b\" (UID: \"461484c0-90e3-4f1a-959a-847f1e9e935b\") "
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.529931    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r72rr\" (UniqueName: \"kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530290    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530315    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530468    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530487    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530501    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af72a3bf-1d16-4cd3-a5a1-74ab07443025-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530516    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qtqt\" (UniqueName: \"kubernetes.io/projected/17009d78-c2f2-496f-b820-2b4765f7b5f1-kube-api-access-4qtqt\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530532    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530546    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530559    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17009d78-c2f2-496f-b820-2b4765f7b5f1-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530572    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af72a3bf-1d16-4cd3-a5a1-74ab07443025-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530585    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6255\" (UniqueName: \"kubernetes.io/projected/af72a3bf-1d16-4cd3-a5a1-74ab07443025-kube-api-access-n6255\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530599    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530613    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.530625    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17009d78-c2f2-496f-b820-2b4765f7b5f1-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.531768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.531338    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca" pod "461484c0-90e3-4f1a-959a-847f1e9e935b" (UID: "461484c0-90e3-4f1a-959a-847f1e9e935b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:26.543044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.534103    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr" pod "461484c0-90e3-4f1a-959a-847f1e9e935b" (UID: "461484c0-90e3-4f1a-959a-847f1e9e935b"). InnerVolumeSpecName "kube-api-access-7txsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:26.543044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.535874    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca" pod "461484c0-90e3-4f1a-959a-847f1e9e935b" (UID: "461484c0-90e3-4f1a-959a-847f1e9e935b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:26.644473 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.644389    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca\") pod \"230f3f79-4978-42c9-89a9-f32eaa09257e\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") "
Oct 31 00:48:26.645008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.644979    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca\") pod \"230f3f79-4978-42c9-89a9-f32eaa09257e\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") "
Oct 31 00:48:26.645245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.645228    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2\" (UniqueName: \"kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2\") pod \"230f3f79-4978-42c9-89a9-f32eaa09257e\" (UID: \"230f3f79-4978-42c9-89a9-f32eaa09257e\") "
Oct 31 00:48:26.646080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.645920    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.646080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.645961    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.646080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.646014    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r72rr\" (UniqueName: \"kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.646326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.646217    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7txsr\" (UniqueName: \"kubernetes.io/projected/461484c0-90e3-4f1a-959a-847f1e9e935b-kube-api-access-7txsr\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.646326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.646237    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.646326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.646251    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/461484c0-90e3-4f1a-959a-847f1e9e935b-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.646940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.646913    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca" pod "230f3f79-4978-42c9-89a9-f32eaa09257e" (UID: "230f3f79-4978-42c9-89a9-f32eaa09257e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:26.648077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.648054    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.652619 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.652575    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca" pod "230f3f79-4978-42c9-89a9-f32eaa09257e" (UID: "230f3f79-4978-42c9-89a9-f32eaa09257e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:26.654237 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.654207    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2" pod "230f3f79-4978-42c9-89a9-f32eaa09257e" (UID: "230f3f79-4978-42c9-89a9-f32eaa09257e"). InnerVolumeSpecName "kube-api-access-q94l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:26.654913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.654886    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.655918 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.655895    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r72rr\" (UniqueName: \"kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr\") pod \"maas-api-revoke-sub-da902600-zxwdt\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.694258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.694219    4924 patch_prober.go:29] interesting pod/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:26.694602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.694286    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:26.755807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.755700    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.755807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.755764    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q94l2\" (UniqueName: \"kubernetes.io/projected/230f3f79-4978-42c9-89a9-f32eaa09257e-kube-api-access-q94l2\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.755807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.755781    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/230f3f79-4978-42c9-89a9-f32eaa09257e-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:26.775829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.774997    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:26.788130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.788094    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"
Oct 31 00:48:26.788792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.788343    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv" event={"ID":"461484c0-90e3-4f1a-959a-847f1e9e935b","Type":"ContainerDied","Data":"c8daa913482376096ee56f040c5f183c5a9aca90f0063065b3e4b81d24180e57"}
Oct 31 00:48:26.788792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.788715    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8daa913482376096ee56f040c5f183c5a9aca90f0063065b3e4b81d24180e57"
Oct 31 00:48:26.792995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.792918    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg" event={"ID":"230f3f79-4978-42c9-89a9-f32eaa09257e","Type":"ContainerDied","Data":"5f7dd335e910559462e567dbb0adfec856e38c4498e6730e9eff48e0ccde3917"}
Oct 31 00:48:26.792995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.792965    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f7dd335e910559462e567dbb0adfec856e38c4498e6730e9eff48e0ccde3917"
Oct 31 00:48:26.793239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.793203    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"
Oct 31 00:48:26.810081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.808444    4924 generic.go:356] "Generic (PLEG): container finished" podID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerID="963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b" exitCode=0
Oct 31 00:48:26.810081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.808681    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"
Oct 31 00:48:26.810081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.809782    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" event={"ID":"af72a3bf-1d16-4cd3-a5a1-74ab07443025","Type":"ContainerDied","Data":"963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b"}
Oct 31 00:48:26.810081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.809847    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" event={"ID":"af72a3bf-1d16-4cd3-a5a1-74ab07443025","Type":"ContainerDied","Data":"15e25e620d8afa38d40511288c6027898b9c1296f17b15812b155bf3af4c9c85"}
Oct 31 00:48:26.810081 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.809876    4924 scope.go:122] "RemoveContainer" containerID="963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b"
Oct 31 00:48:26.818694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.818656    4924 generic.go:356] "Generic (PLEG): container finished" podID="17009d78-c2f2-496f-b820-2b4765f7b5f1" containerID="8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a" exitCode=0
Oct 31 00:48:26.821601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.821439    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" event={"ID":"17009d78-c2f2-496f-b820-2b4765f7b5f1","Type":"ContainerDied","Data":"8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a"}
Oct 31 00:48:26.821601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.821486    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj" event={"ID":"17009d78-c2f2-496f-b820-2b4765f7b5f1","Type":"ContainerDied","Data":"1fe8db3dbf5644196fc96d25f6080bcd9e5a64273d566bbdc44f4250d7f5589f"}
Oct 31 00:48:26.823893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.823868    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"
Oct 31 00:48:26.863479 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.863074    4924 scope.go:122] "RemoveContainer" containerID="963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b"
Oct 31 00:48:26.865491 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:26.865457    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b\": container with ID starting with 963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b not found: ID does not exist" containerID="963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b"
Oct 31 00:48:26.865622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.865503    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b"} err="failed to get container status \"963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b\": rpc error: code = NotFound desc = could not find container \"963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b\": container with ID starting with 963d8878987b52d5c92f5afb466aa942ffb3c8fdcb4a091854a47ba59e24082b not found: ID does not exist"
Oct 31 00:48:26.865622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.865528    4924 scope.go:122] "RemoveContainer" containerID="8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a"
Oct 31 00:48:26.887539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.887508    4924 scope.go:122] "RemoveContainer" containerID="8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a"
Oct 31 00:48:26.888029 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.887967    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"]
Oct 31 00:48:26.894258 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:26.894213    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a\": container with ID starting with 8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a not found: ID does not exist" containerID="8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a"
Oct 31 00:48:26.894368 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.894257    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a"} err="failed to get container status \"8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a\": rpc error: code = NotFound desc = could not find container \"8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a\": container with ID starting with 8d5ede61c188dd8e9a2d262aa79feece19e8451b272cca989fa6d226d3ecff5a not found: ID does not exist"
Oct 31 00:48:26.898401 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.898345    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd"]
Oct 31 00:48:26.937023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.936951    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"]
Oct 31 00:48:26.948833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:26.948761    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7f1496/premium-w1-7f1496-kserve-6f6b97847b-n8kfj"]
Oct 31 00:48:27.135092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.134818    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f1496/distinct-w1-7f1496-kserve-7788ff4bbc-cwbjd" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:27.136646 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:27.136615    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7eedf5_22f2_4482_99ee_0f5a50ec0ff8.slice/crio-957a5da787ef1a6b2a2d29d1ec3a2e6367538e4671de6637837a8a3ed79a8a4c WatchSource:0}: Error finding container 957a5da787ef1a6b2a2d29d1ec3a2e6367538e4671de6637837a8a3ed79a8a4c: Status 404 returned error can't find the container with id 957a5da787ef1a6b2a2d29d1ec3a2e6367538e4671de6637837a8a3ed79a8a4c
Oct 31 00:48:27.139134 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.139075    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"]
Oct 31 00:48:27.836890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.836716    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17009d78-c2f2-496f-b820-2b4765f7b5f1" path="/var/lib/kubelet/pods/17009d78-c2f2-496f-b820-2b4765f7b5f1/volumes"
Oct 31 00:48:27.838500 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.838471    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af72a3bf-1d16-4cd3-a5a1-74ab07443025" path="/var/lib/kubelet/pods/af72a3bf-1d16-4cd3-a5a1-74ab07443025/volumes"
Oct 31 00:48:27.881222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.879929    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt" event={"ID":"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8","Type":"ContainerStarted","Data":"0e3263b05bb8a87a6a21b162dfd54304f606af9b71cbc229e933165d8f832ca4"}
Oct 31 00:48:27.881222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.879996    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt" event={"ID":"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8","Type":"ContainerStarted","Data":"957a5da787ef1a6b2a2d29d1ec3a2e6367538e4671de6637837a8a3ed79a8a4c"}
Oct 31 00:48:27.904412 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:27.904342    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:48:27.923678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:27.923588    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt" podStartSLOduration=2.923546033 podStartE2EDuration="2.923546033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:27.902302707 +0000 UTC m=+908.273086496" watchObservedRunningTime="2026-10-31 00:48:27.923546033 +0000 UTC m=+908.294329822"
Oct 31 00:48:28.403551 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.403482    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506085    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506175    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506217    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506335    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506370    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506520    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506585    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506667    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.508050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.506734    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket\") pod \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\" (UID: \"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819\") "
Oct 31 00:48:28.509499 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.509023    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:28.509499 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.509316    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:28.510224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.509763    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:28.510412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.510383    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:28.511844 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.511818    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:28.524960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.524925    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:28.525241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.525194    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "kube-api-access-7kp7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:28.525978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.525935    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:28.527106 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.527069    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy" pod "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" (UID: "edf6a952-7cc9-4ab5-a6ff-444f8f0ee819"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:28.610350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610306    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kp7s\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-kube-api-access-7kp7s\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610354    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610372    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610386    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610401    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610415    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610429    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610446    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.610594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.610460    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:28.799509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.799459    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:48:28.799612 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.799560    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:48:28.905166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.905125    4924 generic.go:356] "Generic (PLEG): container finished" podID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerID="2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a" exitCode=0
Oct 31 00:48:28.905651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.905201    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"
Oct 31 00:48:28.905651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.905255    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" event={"ID":"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819","Type":"ContainerDied","Data":"2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a"}
Oct 31 00:48:28.905651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.905352    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8" event={"ID":"edf6a952-7cc9-4ab5-a6ff-444f8f0ee819","Type":"ContainerDied","Data":"b40a82424708379dbe1f84b7ec03b2cbbf99e0e537b211a297944cbcf57bd9a4"}
Oct 31 00:48:28.905651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.905380    4924 scope.go:122] "RemoveContainer" containerID="2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a"
Oct 31 00:48:28.937568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.937523    4924 generic.go:356] "Generic (PLEG): container finished" podID="1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" containerID="0e3263b05bb8a87a6a21b162dfd54304f606af9b71cbc229e933165d8f832ca4" exitCode=0
Oct 31 00:48:28.938074 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.938044    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt" event={"ID":"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8","Type":"ContainerDied","Data":"0e3263b05bb8a87a6a21b162dfd54304f606af9b71cbc229e933165d8f832ca4"}
Oct 31 00:48:28.943076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.943040    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx" event={"ID":"ef1215c9-b847-4ddd-81b0-599213179324","Type":"ContainerStarted","Data":"b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1"}
Oct 31 00:48:28.945615 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.945587    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:28.957821 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.957795    4924 scope.go:122] "RemoveContainer" containerID="2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a"
Oct 31 00:48:28.958291 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:28.958262    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a\": container with ID starting with 2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a not found: ID does not exist" containerID="2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a"
Oct 31 00:48:28.959029 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:28.958301    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a"} err="failed to get container status \"2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a\": rpc error: code = NotFound desc = could not find container \"2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a\": container with ID starting with 2f6931efa2fc70ddcbdb18b446ef26340cd291f1956e43322fe6631d2cccbb1a not found: ID does not exist"
Oct 31 00:48:29.020390 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.020335    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx" podStartSLOduration=1.8138541240000001 podStartE2EDuration="21.020320537s" totalImagesPullingTime="19.206466413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:29.010597077 +0000 UTC m=+909.381380869" watchObservedRunningTime="2026-10-31 00:48:29.020320537 +0000 UTC m=+909.391104325"
Oct 31 00:48:29.095639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.095535    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"]
Oct 31 00:48:29.113047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.113005    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-frvc8"]
Oct 31 00:48:29.257768 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.256453    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:48:29.790400 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.790368    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" path="/var/lib/kubelet/pods/edf6a952-7cc9-4ab5-a6ff-444f8f0ee819/volumes"
Oct 31 00:48:29.995794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.992627    4924 generic.go:356] "Generic (PLEG): container finished" podID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerID="0a04fbe3117128c96f38713739570e3490e31a9791d896b12be6eac5a1c67714" exitCode=0
Oct 31 00:48:29.995794 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.992945    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" event={"ID":"3ae8692f-accb-41df-8b30-6955de4d0e5e","Type":"ContainerDied","Data":"0a04fbe3117128c96f38713739570e3490e31a9791d896b12be6eac5a1c67714"}
Oct 31 00:48:29.997970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:29.997070    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:48:30.209434 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.209355    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:48:30.368996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.362211    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w\" (UniqueName: \"kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w\") pod \"3ae8692f-accb-41df-8b30-6955de4d0e5e\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") "
Oct 31 00:48:30.368996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.362427    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume\") pod \"3ae8692f-accb-41df-8b30-6955de4d0e5e\" (UID: \"3ae8692f-accb-41df-8b30-6955de4d0e5e\") "
Oct 31 00:48:30.368996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.363582    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume" pod "3ae8692f-accb-41df-8b30-6955de4d0e5e" (UID: "3ae8692f-accb-41df-8b30-6955de4d0e5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:30.373013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.372950    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w" pod "3ae8692f-accb-41df-8b30-6955de4d0e5e" (UID: "3ae8692f-accb-41df-8b30-6955de4d0e5e"). InnerVolumeSpecName "kube-api-access-mw59w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.458332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.458303    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:30.460392 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.460326    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:48:30.465249 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.465220    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw59w\" (UniqueName: \"kubernetes.io/projected/3ae8692f-accb-41df-8b30-6955de4d0e5e-kube-api-access-mw59w\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.465342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.465253    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ae8692f-accb-41df-8b30-6955de4d0e5e-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.528014 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.527980    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:48:30.565899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.565870    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.566141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.566120    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.566330 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.566311    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.566542 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.566501    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.566725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.566705    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.567003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.566985    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.567506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.567487    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.567651 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.567635    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.567784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.567768    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.567925 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.567909    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca\") pod \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") "
Oct 31 00:48:30.568183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.568167    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert\") pod \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\" (UID: \"ec7ec421-602c-4c8d-9da2-1bf66b7deb55\") "
Oct 31 00:48:30.568319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.568302    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr\" (UniqueName: \"kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr\") pod \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") "
Oct 31 00:48:30.568538 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.568520    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca\") pod \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\" (UID: \"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8\") "
Oct 31 00:48:30.570235 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.570212    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.571317 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.567323    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.571389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.569136    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:30.571389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.569251    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.571389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.569424    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.571389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.569627    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:30.571389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.569921    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca" pod "1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" (UID: "1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:30.572492 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.572461    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca" pod "1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" (UID: "1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:30.591812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.591778    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "kube-api-access-m6d8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.591812 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.591772    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.595367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.595102    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token" pod "ec7ec421-602c-4c8d-9da2-1bf66b7deb55" (UID: "ec7ec421-602c-4c8d-9da2-1bf66b7deb55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.595833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.595804    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr" pod "1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" (UID: "1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8"). InnerVolumeSpecName "kube-api-access-r72rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.599142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.599117    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:48:30.670930 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.670895    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.670966    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671006    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671044    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671096    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671128    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671154    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671180    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671216    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671273    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671332    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671677 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671398    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671677 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671429    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671677 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671462    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671677 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671555    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671874    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket\") pod \"721f7484-545b-4792-9f79-0dfa23a82962\" (UID: \"721f7484-545b-4792-9f79-0dfa23a82962\") "
Oct 31 00:48:30.671981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671916    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.671981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.671960    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token\") pod \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\" (UID: \"aee1ad5b-ce6d-4946-a3f6-01a933e58d07\") "
Oct 31 00:48:30.672913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.672456    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.672913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.672456    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.672913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.672638    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.672913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.672657    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.672913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.672867    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.673270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.672988    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:30.673270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673010    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.673270 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673239    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.673498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673472    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673557    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673711    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6d8t\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-kube-api-access-m6d8t\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673739    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673773    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673789    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673803    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673817    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673833    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673846    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673860    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r72rr\" (UniqueName: \"kubernetes.io/projected/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-kube-api-access-r72rr\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673874    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673886    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673900    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673913    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673927    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673940    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673954    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673968    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721f7484-545b-4792-9f79-0dfa23a82962-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673982    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.673997    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.674011    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec7ec421-602c-4c8d-9da2-1bf66b7deb55-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.675055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.674024    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.676228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.675963    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.676803 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.676761    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "kube-api-access-t9jdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.677242 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.677206    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.677484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.677457    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.677580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.677536    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:30.678055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.678017    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:30.678644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.678621    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt" pod "aee1ad5b-ce6d-4946-a3f6-01a933e58d07" (UID: "aee1ad5b-ce6d-4946-a3f6-01a933e58d07"). InnerVolumeSpecName "kube-api-access-lt6wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:30.678805 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.678737    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo" pod "721f7484-545b-4792-9f79-0dfa23a82962" (UID: "721f7484-545b-4792-9f79-0dfa23a82962"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:30.776011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.775977    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776006    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721f7484-545b-4792-9f79-0dfa23a82962-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776017    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt6wt\" (UniqueName: \"kubernetes.io/projected/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-kube-api-access-lt6wt\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776028    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9jdx\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-kube-api-access-t9jdx\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776037    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776046    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721f7484-545b-4792-9f79-0dfa23a82962-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776054    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721f7484-545b-4792-9f79-0dfa23a82962-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:30.776333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:30.776063    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1ad5b-ce6d-4946-a3f6-01a933e58d07-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:31.013007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.012972    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"
Oct 31 00:48:31.013007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.012983    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt" event={"ID":"1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8","Type":"ContainerDied","Data":"957a5da787ef1a6b2a2d29d1ec3a2e6367538e4671de6637837a8a3ed79a8a4c"}
Oct 31 00:48:31.013562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.013074    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="957a5da787ef1a6b2a2d29d1ec3a2e6367538e4671de6637837a8a3ed79a8a4c"
Oct 31 00:48:31.014839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.014806    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb" event={"ID":"3ae8692f-accb-41df-8b30-6955de4d0e5e","Type":"ContainerDied","Data":"a7b704b4cfedd2c34776d44216ffc91f24445fbf4ffa8b94b573c4493837fbdb"}
Oct 31 00:48:31.014963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.014850    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"
Oct 31 00:48:31.014963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.014879    4924 scope.go:122] "RemoveContainer" containerID="0a04fbe3117128c96f38713739570e3490e31a9791d896b12be6eac5a1c67714"
Oct 31 00:48:31.016970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.016685    4924 generic.go:356] "Generic (PLEG): container finished" podID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerID="90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15" exitCode=0
Oct 31 00:48:31.016970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.016883    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" event={"ID":"ec7ec421-602c-4c8d-9da2-1bf66b7deb55","Type":"ContainerDied","Data":"90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15"}
Oct 31 00:48:31.016970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.016927    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh" event={"ID":"ec7ec421-602c-4c8d-9da2-1bf66b7deb55","Type":"ContainerDied","Data":"01bfca5fb46bea92fa69f7ae4b04c70edd7047e8209e77bcdc27b1fd0a979cbf"}
Oct 31 00:48:31.017152 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.016990    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"
Oct 31 00:48:31.025208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.023831    4924 generic.go:356] "Generic (PLEG): container finished" podID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerID="56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42" exitCode=0
Oct 31 00:48:31.025208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.024005    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" event={"ID":"aee1ad5b-ce6d-4946-a3f6-01a933e58d07","Type":"ContainerDied","Data":"56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42"}
Oct 31 00:48:31.025208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.024049    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq" event={"ID":"aee1ad5b-ce6d-4946-a3f6-01a933e58d07","Type":"ContainerDied","Data":"380e1592fd988f90bd4064323af7f1deb3a4f77de09ae1dc5e1f3be83b3a010c"}
Oct 31 00:48:31.025208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.024145    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"
Oct 31 00:48:31.029207 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.029161    4924 generic.go:356] "Generic (PLEG): container finished" podID="721f7484-545b-4792-9f79-0dfa23a82962" containerID="9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f" exitCode=0
Oct 31 00:48:31.029344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.029326    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"
Oct 31 00:48:31.029576 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.029323    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" event={"ID":"721f7484-545b-4792-9f79-0dfa23a82962","Type":"ContainerDied","Data":"9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f"}
Oct 31 00:48:31.029932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.029911    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb" event={"ID":"721f7484-545b-4792-9f79-0dfa23a82962","Type":"ContainerDied","Data":"8c35af4ec95d437016683493695520a052d1106809b7fbe425be4cc82ce1da31"}
Oct 31 00:48:31.035929 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.035852    4924 scope.go:122] "RemoveContainer" containerID="90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15"
Oct 31 00:48:31.053344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.053320    4924 scope.go:122] "RemoveContainer" containerID="90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15"
Oct 31 00:48:31.053852 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:31.053825    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15\": container with ID starting with 90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15 not found: ID does not exist" containerID="90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15"
Oct 31 00:48:31.053923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.053866    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15"} err="failed to get container status \"90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15\": rpc error: code = NotFound desc = could not find container \"90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15\": container with ID starting with 90307948832e0b01fb849aa3c954c9d5d2eea25e7e4c5f18a4e4f7425e563f15 not found: ID does not exist"
Oct 31 00:48:31.053923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.053893    4924 scope.go:122] "RemoveContainer" containerID="56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42"
Oct 31 00:48:31.058360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.058332    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"]
Oct 31 00:48:31.066986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.066955    4924 scope.go:122] "RemoveContainer" containerID="56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42"
Oct 31 00:48:31.067336 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:31.067316    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42\": container with ID starting with 56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42 not found: ID does not exist" containerID="56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42"
Oct 31 00:48:31.067407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.067344    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42"} err="failed to get container status \"56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42\": rpc error: code = NotFound desc = could not find container \"56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42\": container with ID starting with 56ffcfe7031e8fc97344c48ad8aa929bcb45376b8136c4f95e2344caaa79eb42 not found: ID does not exist"
Oct 31 00:48:31.067407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.067363    4924 scope.go:122] "RemoveContainer" containerID="9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f"
Oct 31 00:48:31.068828 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.068802    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-lhvsh"]
Oct 31 00:48:31.078326 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.078291    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"]
Oct 31 00:48:31.081459 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.081435    4924 scope.go:122] "RemoveContainer" containerID="e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"
Oct 31 00:48:31.088669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.088641    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-47f370-d6694f97-xt8tb"]
Oct 31 00:48:31.094992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.094959    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"]
Oct 31 00:48:31.100814 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.100783    4924 scope.go:122] "RemoveContainer" containerID="9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f"
Oct 31 00:48:31.101190 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:31.101168    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f\": container with ID starting with 9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f not found: ID does not exist" containerID="9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f"
Oct 31 00:48:31.101266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.101198    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f"} err="failed to get container status \"9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f\": rpc error: code = NotFound desc = could not find container \"9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f\": container with ID starting with 9e7edf6f230884b1b434d9972c7dde7701b11ce5b70a287adca5e5adf09e1c6f not found: ID does not exist"
Oct 31 00:48:31.101266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.101217    4924 scope.go:122] "RemoveContainer" containerID="e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"
Oct 31 00:48:31.101512 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:31.101488    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9\": container with ID starting with e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9 not found: ID does not exist" containerID="e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"
Oct 31 00:48:31.101573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.101518    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9"} err="failed to get container status \"e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9\": rpc error: code = NotFound desc = could not find container \"e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9\": container with ID starting with e1a81252e6bde36a84990d2538f9460bd3f497ec955190a131b06144d07d81f9 not found: ID does not exist"
Oct 31 00:48:31.101631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.101612    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-66zwb"]
Oct 31 00:48:31.107912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.107869    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"]
Oct 31 00:48:31.114247 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.114218    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-47f370-openshift-default-6dc8794f9-75fsq"]
Oct 31 00:48:31.268222 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.268187    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:48:31.766273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.766246    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" path="/var/lib/kubelet/pods/3ae8692f-accb-41df-8b30-6955de4d0e5e/volumes"
Oct 31 00:48:31.766827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.766813    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="721f7484-545b-4792-9f79-0dfa23a82962" path="/var/lib/kubelet/pods/721f7484-545b-4792-9f79-0dfa23a82962/volumes"
Oct 31 00:48:31.767293 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.767281    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" path="/var/lib/kubelet/pods/aee1ad5b-ce6d-4946-a3f6-01a933e58d07/volumes"
Oct 31 00:48:31.768132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:31.768116    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" path="/var/lib/kubelet/pods/ec7ec421-602c-4c8d-9da2-1bf66b7deb55/volumes"
Oct 31 00:48:32.356594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.356557    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"]
Oct 31 00:48:32.357268 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357250    4924 cpu_manager.go:414] "containerMap: removing container" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerName="payload-processing"
Oct 31 00:48:32.357310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357274    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerName="payload-processing"
Oct 31 00:48:32.357310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357286    4924 cpu_manager.go:414] "containerMap: removing container" podUID="1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" containerName="revoke-keys"
Oct 31 00:48:32.357310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357291    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" containerName="revoke-keys"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357320    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357326    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357336    4924 cpu_manager.go:414] "containerMap: removing container" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357342    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357350    4924 cpu_manager.go:414] "containerMap: removing container" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357354    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357365    4924 cpu_manager.go:414] "containerMap: removing container" podUID="230f3f79-4978-42c9-89a9-f32eaa09257e" containerName="revoke-keys"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357369    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="230f3f79-4978-42c9-89a9-f32eaa09257e" containerName="revoke-keys"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357401    4924 cpu_manager.go:414] "containerMap: removing container" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy"
Oct 31 00:48:32.357415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357409    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357427    4924 cpu_manager.go:414] "containerMap: removing container" podUID="461484c0-90e3-4f1a-959a-847f1e9e935b" containerName="revoke-keys"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357431    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="461484c0-90e3-4f1a-959a-847f1e9e935b" containerName="revoke-keys"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357548    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="aee1ad5b-ce6d-4946-a3f6-01a933e58d07" containerName="istio-proxy"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357557    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357565    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="461484c0-90e3-4f1a-959a-847f1e9e935b" containerName="revoke-keys"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357573    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ae8692f-accb-41df-8b30-6955de4d0e5e" containerName="payload-processing"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357578    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" containerName="revoke-keys"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357585    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec7ec421-602c-4c8d-9da2-1bf66b7deb55" containerName="istio-proxy"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357590    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357597    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="edf6a952-7cc9-4ab5-a6ff-444f8f0ee819" containerName="istio-proxy"
Oct 31 00:48:32.357698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.357604    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="230f3f79-4978-42c9-89a9-f32eaa09257e" containerName="revoke-keys"
Oct 31 00:48:32.376789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.364867    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.376789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.371133    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"]
Oct 31 00:48:32.376789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.374819    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:48:32.376789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.375800    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:48:32.533066 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.533016    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.533288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.533267    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rngxq\" (UniqueName: \"kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.533388 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.533372    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.634683 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.634579    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.634870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.634716    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rngxq\" (UniqueName: \"kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.634870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.634795    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.635445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.635410    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.637500 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.637468    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.647290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.647245    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rngxq\" (UniqueName: \"kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq\") pod \"maas-api-revoke-sub-5460bdd0-nbt92\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:32.706177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:32.706129    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:33.012591 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:33.012551    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0ae94c_fbfe_4259_8b61_5dd9e1b10398.slice/crio-6beb7f9a90796818518e82593d2e1dae6c6f433aa1d672e2e8f4927b0802b27c WatchSource:0}: Error finding container 6beb7f9a90796818518e82593d2e1dae6c6f433aa1d672e2e8f4927b0802b27c: Status 404 returned error can't find the container with id 6beb7f9a90796818518e82593d2e1dae6c6f433aa1d672e2e8f4927b0802b27c
Oct 31 00:48:33.013224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:33.013198    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"]
Oct 31 00:48:33.057514 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:33.057476    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92" event={"ID":"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398","Type":"ContainerStarted","Data":"6beb7f9a90796818518e82593d2e1dae6c6f433aa1d672e2e8f4927b0802b27c"}
Oct 31 00:48:34.085829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:34.081783    4924 generic.go:356] "Generic (PLEG): container finished" podID="3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" containerID="deba036cf0e6d4f92f4a7bc0b2f219c653c20f40dd25c5c43b9442031ffb2eb7" exitCode=0
Oct 31 00:48:34.085829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:34.081931    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92" event={"ID":"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398","Type":"ContainerDied","Data":"deba036cf0e6d4f92f4a7bc0b2f219c653c20f40dd25c5c43b9442031ffb2eb7"}
Oct 31 00:48:35.328513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.328483    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:35.378650 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.378472    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca\") pod \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") "
Oct 31 00:48:35.378650 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.378621    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq\" (UniqueName: \"kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq\") pod \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") "
Oct 31 00:48:35.378916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.378669    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca\") pod \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\" (UID: \"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398\") "
Oct 31 00:48:35.379579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.379496    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca" pod "3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" (UID: "3c0ae94c-fbfe-4259-8b61-5dd9e1b10398"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:35.381623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.381594    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq" pod "3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" (UID: "3c0ae94c-fbfe-4259-8b61-5dd9e1b10398"). InnerVolumeSpecName "kube-api-access-rngxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:35.382228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.382209    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca" pod "3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" (UID: "3c0ae94c-fbfe-4259-8b61-5dd9e1b10398"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:35.483515 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.483474    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:35.483515 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.483517    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rngxq\" (UniqueName: \"kubernetes.io/projected/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-kube-api-access-rngxq\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:35.484034 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.483534    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:35.566707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.566665    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"]
Oct 31 00:48:35.567050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.567016    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy" containerID="cri-o://8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf" gracePeriod=30
Oct 31 00:48:35.590312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.590268    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"]
Oct 31 00:48:35.590626 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.590594    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" containerID="cri-o://287bf48c0c475dababc89afbbf8cbdccfe36e327e6558ef4ece9dfb96f41adf5" gracePeriod=30
Oct 31 00:48:35.612820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.612083    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"]
Oct 31 00:48:35.612820 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.612423    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" containerID="cri-o://1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51" gracePeriod=30
Oct 31 00:48:35.660200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.660103    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"]
Oct 31 00:48:35.660874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.660550    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" containerID="cri-o://35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940" gracePeriod=30
Oct 31 00:48:35.668279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.668171    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"]
Oct 31 00:48:35.668597 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.668535    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" containerID="cri-o://6863e752021b11433718bd0183023e98982d57b28fb8db0f7446e8c571554cfd" gracePeriod=30
Oct 31 00:48:35.936600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.936506    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:35.936600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:35.936559    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:36.033929 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.033888    4924 patch_prober.go:29] interesting pod/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:36.034093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.033961    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:36.103882 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.103853    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"
Oct 31 00:48:36.104133 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.103885    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92" event={"ID":"3c0ae94c-fbfe-4259-8b61-5dd9e1b10398","Type":"ContainerDied","Data":"6beb7f9a90796818518e82593d2e1dae6c6f433aa1d672e2e8f4927b0802b27c"}
Oct 31 00:48:36.104133 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.103940    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6beb7f9a90796818518e82593d2e1dae6c6f433aa1d672e2e8f4927b0802b27c"
Oct 31 00:48:36.539140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.539102    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:36.539568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:36.539202    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:37.814933 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.814896    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"]
Oct 31 00:48:37.815799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.815735    4924 cpu_manager.go:414] "containerMap: removing container" podUID="3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" containerName="revoke-keys"
Oct 31 00:48:37.815799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.815779    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" containerName="revoke-keys"
Oct 31 00:48:37.815972 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.815827    4924 cpu_manager.go:414] "containerMap: removing container" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy"
Oct 31 00:48:37.815972 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.815835    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="721f7484-545b-4792-9f79-0dfa23a82962" containerName="istio-proxy"
Oct 31 00:48:37.816038 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.815995    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" containerName="revoke-keys"
Oct 31 00:48:37.819957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.819933    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:37.824315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.824291    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:48:37.824451 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.824426    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:48:37.833708 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.833680    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"]
Oct 31 00:48:37.933777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.933704    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmcfv\" (UniqueName: \"kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:37.933993 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.933968    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:37.934084 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:37.934012    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.035957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.035908    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmcfv\" (UniqueName: \"kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.036115 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.035975    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.036115 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.036019    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.036716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.036688    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.038459 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.038437    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.046558 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.046522    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmcfv\" (UniqueName: \"kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv\") pod \"maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.140197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.140106    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:38.514731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:38.514696    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"]
Oct 31 00:48:39.142215 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.142104    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq" event={"ID":"0e783ee8-1151-4009-b889-de32cb744cbc","Type":"ContainerStarted","Data":"bb9881a0b47edae0588ffb57c74795a9dfc197ee3557195b5893983619100e1f"}
Oct 31 00:48:39.142775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.142513    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq" event={"ID":"0e783ee8-1151-4009-b889-de32cb744cbc","Type":"ContainerStarted","Data":"dabe3ace709a7f52524a3b3c49c21664c075e48ed4597381770f7ffdc03fcc43"}
Oct 31 00:48:39.162217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.162168    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq" podStartSLOduration=2.162154181 podStartE2EDuration="2.162154181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:39.159041858 +0000 UTC m=+919.529825671" watchObservedRunningTime="2026-10-31 00:48:39.162154181 +0000 UTC m=+919.532937970"
Oct 31 00:48:39.581246 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.581202    4924 patch_prober.go:29] interesting pod/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:39.581405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.581282    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:39.930953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.930884    4924 pod_container_manager_linux.go:218] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","podfb0c9073-46bd-4718-8d47-84a00663c82d"] err="unable to destroy cgroup paths for cgroup [kubepods burstable podfb0c9073-46bd-4718-8d47-84a00663c82d] : Timed out while waiting for systemd to remove kubepods-burstable-podfb0c9073_46bd_4718_8d47_84a00663c82d.slice"
Oct 31 00:48:39.930953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.930937    4924 kubelet_pods.go:2797] "Failed to destroy orphaned pod cgroup" podUID="fb0c9073-46bd-4718-8d47-84a00663c82d" err="failed to delete cgroup paths for [kubepods burstable podfb0c9073-46bd-4718-8d47-84a00663c82d] : unable to destroy cgroup paths for cgroup [kubepods burstable podfb0c9073-46bd-4718-8d47-84a00663c82d] : Timed out while waiting for systemd to remove kubepods-burstable-podfb0c9073_46bd_4718_8d47_84a00663c82d.slice"
Oct 31 00:48:39.971599 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.971563    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"]
Oct 31 00:48:39.972644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:39.972622    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"]
Oct 31 00:48:40.011834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.006080    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.056815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.055501    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"]
Oct 31 00:48:40.189370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.189280    4924 generic.go:356] "Generic (PLEG): container finished" podID="0e783ee8-1151-4009-b889-de32cb744cbc" containerID="bb9881a0b47edae0588ffb57c74795a9dfc197ee3557195b5893983619100e1f" exitCode=0
Oct 31 00:48:40.190342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.190296    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq" event={"ID":"0e783ee8-1151-4009-b889-de32cb744cbc","Type":"ContainerDied","Data":"bb9881a0b47edae0588ffb57c74795a9dfc197ee3557195b5893983619100e1f"}
Oct 31 00:48:40.210856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.210812    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.215942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.215906    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrnvs\" (UniqueName: \"kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.216397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.216378    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.326838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.317724    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.326838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.325577    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.326838 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.325704    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrnvs\" (UniqueName: \"kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.327525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.327276    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.347202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.347134    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrnvs\" (UniqueName: \"kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.358529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.358493    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ddaa97b6-5g6fc\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.380837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.380795    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:40.868275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:40.852291    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"]
Oct 31 00:48:41.135782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.135327    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:48:41.147703 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.147628    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:48:41.154457 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.154353    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:48:41.231987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.231953    4924 generic.go:356] "Generic (PLEG): container finished" podID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerID="287bf48c0c475dababc89afbbf8cbdccfe36e327e6558ef4ece9dfb96f41adf5" exitCode=0
Oct 31 00:48:41.232414 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.232050    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" event={"ID":"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5","Type":"ContainerDied","Data":"287bf48c0c475dababc89afbbf8cbdccfe36e327e6558ef4ece9dfb96f41adf5"}
Oct 31 00:48:41.235125 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.235095    4924 generic.go:356] "Generic (PLEG): container finished" podID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerID="1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51" exitCode=0
Oct 31 00:48:41.235475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.235451    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" event={"ID":"8cb74ba3-f886-4ee9-80e6-d85a149092a7","Type":"ContainerDied","Data":"1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51"}
Oct 31 00:48:41.235654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.235635    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb" event={"ID":"8cb74ba3-f886-4ee9-80e6-d85a149092a7","Type":"ContainerDied","Data":"b61b6e171439c2bdc25010834f565a02304c7598d1be7b86b7089a4996f9893b"}
Oct 31 00:48:41.235796 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.235781    4924 scope.go:122] "RemoveContainer" containerID="1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51"
Oct 31 00:48:41.236070 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.236053    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"
Oct 31 00:48:41.237266 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.236911    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:48:41.244302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.241644    4924 generic.go:356] "Generic (PLEG): container finished" podID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerID="8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf" exitCode=0
Oct 31 00:48:41.244302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.243834    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" event={"ID":"afdebed2-b2b3-4856-9af3-13a739db3fdc","Type":"ContainerDied","Data":"8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf"}
Oct 31 00:48:41.244302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.243890    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x" event={"ID":"afdebed2-b2b3-4856-9af3-13a739db3fdc","Type":"ContainerDied","Data":"914b9789a2cab2db4bad8c5f0ee4b58bfcae0f3bbebed5fe7a1bbb2c2bc54091"}
Oct 31 00:48:41.244302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.243965    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"
Oct 31 00:48:41.251716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.251600    4924 generic.go:356] "Generic (PLEG): container finished" podID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerID="bb30d968d5bd5e02cbe1d3f6c6ce96a33498f77f26816a371347b06a97497487" exitCode=0
Oct 31 00:48:41.251966 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.251944    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw" event={"ID":"54fe85ab-8589-424f-aea2-4b5fe867ca4c","Type":"ContainerDied","Data":"bb30d968d5bd5e02cbe1d3f6c6ce96a33498f77f26816a371347b06a97497487"}
Oct 31 00:48:41.252165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.252149    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"
Oct 31 00:48:41.254271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.254251    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:48:41.255085 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.255057    4924 scope.go:122] "RemoveContainer" containerID="1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51"
Oct 31 00:48:41.256510 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:41.256481    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51\": container with ID starting with 1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51 not found: ID does not exist" containerID="1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51"
Oct 31 00:48:41.256574 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.256522    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51"} err="failed to get container status \"1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51\": rpc error: code = NotFound desc = could not find container \"1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51\": container with ID starting with 1536b6c175e78b912af6dd067628459313d959d94d6f4dc59a6149f07f9edd51 not found: ID does not exist"
Oct 31 00:48:41.256574 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.256546    4924 scope.go:122] "RemoveContainer" containerID="8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf"
Oct 31 00:48:41.259497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.259406    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" event={"ID":"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30","Type":"ContainerDied","Data":"6863e752021b11433718bd0183023e98982d57b28fb8db0f7446e8c571554cfd"}
Oct 31 00:48:41.259672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.259301    4924 generic.go:356] "Generic (PLEG): container finished" podID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerID="6863e752021b11433718bd0183023e98982d57b28fb8db0f7446e8c571554cfd" exitCode=0
Oct 31 00:48:41.261080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.261059    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.270107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.270086    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.270574 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.270556    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.270992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.270975    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.271532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.267071    4924 generic.go:356] "Generic (PLEG): container finished" podID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerID="35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940" exitCode=0
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.267264    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.261285    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.261610    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.270912    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.262960    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc" event={"ID":"7945106c-574a-4b09-8d93-4366e9d0aa07","Type":"ContainerStarted","Data":"05fb53da59066fd30fd9bd6251de77c9e56d184a9aaa2598558db4c4c795baa0"}
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.273407    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" event={"ID":"6f0ad9de-782c-41ae-87af-ea375f20c260","Type":"ContainerDied","Data":"35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940"}
Oct 31 00:48:41.273482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.273450    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77" event={"ID":"6f0ad9de-782c-41ae-87af-ea375f20c260","Type":"ContainerDied","Data":"af4bee1744a7648c76ecb9e582df701542a6b7b39c76cb33f277f8db2dcb97b3"}
Oct 31 00:48:41.278469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278434    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.278564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278524    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.278564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278554    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.278676 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278592    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.278676 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278637    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.278810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278700    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.278810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278726    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.278810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278794    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.278957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278822    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.278957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278858    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.278957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278889    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.278957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278920    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.279166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.278955    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.279166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279007    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.279166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279036    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.279166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279086    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.279166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279133    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.279166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279158    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket\") pod \"6f0ad9de-782c-41ae-87af-ea375f20c260\" (UID: \"6f0ad9de-782c-41ae-87af-ea375f20c260\") "
Oct 31 00:48:41.279472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279190    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.279472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279308    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.279472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279355    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.279472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279410    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f\") pod \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\" (UID: \"8cb74ba3-f886-4ee9-80e6-d85a149092a7\") "
Oct 31 00:48:41.279472 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.279467    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket\") pod \"afdebed2-b2b3-4856-9af3-13a739db3fdc\" (UID: \"afdebed2-b2b3-4856-9af3-13a739db3fdc\") "
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280048    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280422    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280447    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280462    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f0ad9de-782c-41ae-87af-ea375f20c260-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280486    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280791    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.281534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.280841    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.282547 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.282276    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.284850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.283691    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.284850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.284074    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.284850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.284277    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.284850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.284496    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.285652 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.285619    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.286263 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.286239    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.289435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.289416    4924 scope.go:122] "RemoveContainer" containerID="aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"
Oct 31 00:48:41.291724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.291698    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.297847 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.297819    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.298217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.298161    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "kube-api-access-9glgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.298217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.298184    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:41.307544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.307345    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.307544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.307425    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:41.307544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.307518    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.330810    4924 scope.go:122] "RemoveContainer" containerID="8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf"
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:41.331294    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf\": container with ID starting with 8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf not found: ID does not exist" containerID="8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf"
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.331336    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf"} err="failed to get container status \"8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf\": rpc error: code = NotFound desc = could not find container \"8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf\": container with ID starting with 8f3efc36bc779e51b2c395d3799bfc68344fae3c8410eb16e415dabd5d18cfaf not found: ID does not exist"
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.331363    4924 scope.go:122] "RemoveContainer" containerID="aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:41.332109    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f\": container with ID starting with aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f not found: ID does not exist" containerID="aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.332144    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f"} err="failed to get container status \"aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f\": rpc error: code = NotFound desc = could not find container \"aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f\": container with ID starting with aaee29630c6a24fe7b17133c95a68ab79d20929c335e00ce489d41c91543342f not found: ID does not exist"
Oct 31 00:48:41.333583 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.332168    4924 scope.go:122] "RemoveContainer" containerID="bb30d968d5bd5e02cbe1d3f6c6ce96a33498f77f26816a371347b06a97497487"
Oct 31 00:48:41.382840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.382738    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.382840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.382820    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.383077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.382853    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.383077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.382887    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.383077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.382942    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.383077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.382973    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.383077 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.383062    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.383341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.383148    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.383341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.383174    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.383341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.383267    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.383341 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.383296    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384375    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384533    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384582    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume\") pod \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384584    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384674    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384703    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384740    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384796    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384852    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn\") pod \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\" (UID: \"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30\") "
Oct 31 00:48:41.384873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384881    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.385546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.384916    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh\" (UniqueName: \"kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh\") pod \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\" (UID: \"54fe85ab-8589-424f-aea2-4b5fe867ca4c\") "
Oct 31 00:48:41.385546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.385026    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo\") pod \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\" (UID: \"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5\") "
Oct 31 00:48:41.385546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.385151    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume" pod "54fe85ab-8589-424f-aea2-4b5fe867ca4c" (UID: "54fe85ab-8589-424f-aea2-4b5fe867ca4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.385757 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.385653    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.386092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.385935    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.386181 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.386116    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.386504 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.386480    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.390060 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390032    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390064    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390080    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afdebed2-b2b3-4856-9af3-13a739db3fdc-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390093    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390107    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390120    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390134    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fe85ab-8589-424f-aea2-4b5fe867ca4c-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390148    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390162    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390175    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390190    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390204    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390217    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.390580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.390233    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391321    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391585    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391609    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391625    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f0ad9de-782c-41ae-87af-ea375f20c260-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391649    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391664    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391678    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391692    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391706    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391915    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391931    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.392000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.391945    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9glgr\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-kube-api-access-9glgr\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.393207 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.392376    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.393207 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.392429    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.393321 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.393218    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.393816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.393690    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.393816 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.393773    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.394461 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.394346    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token" pod "6f0ad9de-782c-41ae-87af-ea375f20c260" (UID: "6f0ad9de-782c-41ae-87af-ea375f20c260"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.394461 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.394420    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f" pod "8cb74ba3-f886-4ee9-80e6-d85a149092a7" (UID: "8cb74ba3-f886-4ee9-80e6-d85a149092a7"). InnerVolumeSpecName "kube-api-access-67s4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.394461 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.394441    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.403793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.396541    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l" pod "afdebed2-b2b3-4856-9af3-13a739db3fdc" (UID: "afdebed2-b2b3-4856-9af3-13a739db3fdc"). InnerVolumeSpecName "kube-api-access-nfz5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.403793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.397952    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "kube-api-access-x5lfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.403793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.400041    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:41.403793 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.403634    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "kube-api-access-7npdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.409225 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.409187    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh" pod "54fe85ab-8589-424f-aea2-4b5fe867ca4c" (UID: "54fe85ab-8589-424f-aea2-4b5fe867ca4c"). InnerVolumeSpecName "kube-api-access-b2phh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.412719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.412583    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.412719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.412584    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.412719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.412693    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:41.413212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.412983    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token" pod "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" (UID: "5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.416025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.415660    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo" pod "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" (UID: "fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497870    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497910    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f0ad9de-782c-41ae-87af-ea375f20c260-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497927    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7npdn\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-kube-api-access-7npdn\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497943    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497959    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497973    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.497986    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498001    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5lfn\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-kube-api-access-x5lfn\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498017    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498031    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2phh\" (UniqueName: \"kubernetes.io/projected/54fe85ab-8589-424f-aea2-4b5fe867ca4c-kube-api-access-b2phh\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498045    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498060    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfz5l\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-kube-api-access-nfz5l\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498074    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498087    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498100    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498113    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67s4f\" (UniqueName: \"kubernetes.io/projected/8cb74ba3-f886-4ee9-80e6-d85a149092a7-kube-api-access-67s4f\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498157    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498176    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.498998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.498189    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afdebed2-b2b3-4856-9af3-13a739db3fdc-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.506539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.506510    4924 scope.go:122] "RemoveContainer" containerID="35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940"
Oct 31 00:48:41.524840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.524672    4924 scope.go:122] "RemoveContainer" containerID="35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940"
Oct 31 00:48:41.525168 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:41.525094    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940\": container with ID starting with 35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940 not found: ID does not exist" containerID="35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940"
Oct 31 00:48:41.525168 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.525129    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940"} err="failed to get container status \"35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940\": rpc error: code = NotFound desc = could not find container \"35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940\": container with ID starting with 35ef4d072693fa537f6e9a9b5989289836a1fb5e1ee432e07807b53eeb529940 not found: ID does not exist"
Oct 31 00:48:41.587353 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.584532    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"]
Oct 31 00:48:41.591737 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.591273    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-v8vcb"]
Oct 31 00:48:41.611493 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.611461    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"]
Oct 31 00:48:41.619016 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.618987    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f1496-6f98bd898c-kb6sw"]
Oct 31 00:48:41.634903 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.634811    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:41.636570 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.636544    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"]
Oct 31 00:48:41.642587 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.642564    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-ks68x"]
Oct 31 00:48:41.666531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.666489    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"]
Oct 31 00:48:41.673937 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.673910    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-4mh77"]
Oct 31 00:48:41.717567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.717515    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv\" (UniqueName: \"kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv\") pod \"0e783ee8-1151-4009-b889-de32cb744cbc\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") "
Oct 31 00:48:41.717819 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.717768    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca\") pod \"0e783ee8-1151-4009-b889-de32cb744cbc\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") "
Oct 31 00:48:41.717969 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.717895    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca\") pod \"0e783ee8-1151-4009-b889-de32cb744cbc\" (UID: \"0e783ee8-1151-4009-b889-de32cb744cbc\") "
Oct 31 00:48:41.721880 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.721829    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca" pod "0e783ee8-1151-4009-b889-de32cb744cbc" (UID: "0e783ee8-1151-4009-b889-de32cb744cbc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:41.729275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.729239    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv" pod "0e783ee8-1151-4009-b889-de32cb744cbc" (UID: "0e783ee8-1151-4009-b889-de32cb744cbc"). InnerVolumeSpecName "kube-api-access-lmcfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:41.734053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.734003    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca" pod "0e783ee8-1151-4009-b889-de32cb744cbc" (UID: "0e783ee8-1151-4009-b889-de32cb744cbc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:41.768999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.768959    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" path="/var/lib/kubelet/pods/54fe85ab-8589-424f-aea2-4b5fe867ca4c/volumes"
Oct 31 00:48:41.769809 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.769779    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" path="/var/lib/kubelet/pods/6f0ad9de-782c-41ae-87af-ea375f20c260/volumes"
Oct 31 00:48:41.770412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.770393    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" path="/var/lib/kubelet/pods/8cb74ba3-f886-4ee9-80e6-d85a149092a7/volumes"
Oct 31 00:48:41.771151 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.771133    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" path="/var/lib/kubelet/pods/afdebed2-b2b3-4856-9af3-13a739db3fdc/volumes"
Oct 31 00:48:41.820230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.820189    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.820230 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.820228    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e783ee8-1151-4009-b889-de32cb744cbc-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.820452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.820241    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmcfv\" (UniqueName: \"kubernetes.io/projected/0e783ee8-1151-4009-b889-de32cb744cbc-kube-api-access-lmcfv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:41.895110 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:41.895023    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main" containerID="cri-o://7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013" gracePeriod=45
Oct 31 00:48:42.231450 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.231415    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:48:42.305689 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.305645    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"
Oct 31 00:48:42.306302 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.305646    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq" event={"ID":"0e783ee8-1151-4009-b889-de32cb744cbc","Type":"ContainerDied","Data":"dabe3ace709a7f52524a3b3c49c21664c075e48ed4597381770f7ffdc03fcc43"}
Oct 31 00:48:42.306465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.306439    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dabe3ace709a7f52524a3b3c49c21664c075e48ed4597381770f7ffdc03fcc43"
Oct 31 00:48:42.315345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.315309    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w" event={"ID":"5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5","Type":"ContainerDied","Data":"062cf78846ace9eafab358cba52819b310a1a4b3b31d9db06cd2b4e1452ef9ff"}
Oct 31 00:48:42.315475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.315385    4924 scope.go:122] "RemoveContainer" containerID="287bf48c0c475dababc89afbbf8cbdccfe36e327e6558ef4ece9dfb96f41adf5"
Oct 31 00:48:42.315475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.315323    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"
Oct 31 00:48:42.319533 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.319505    4924 generic.go:356] "Generic (PLEG): container finished" podID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerID="7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013" exitCode=0
Oct 31 00:48:42.319653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.319614    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"
Oct 31 00:48:42.319716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.319615    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" event={"ID":"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5","Type":"ContainerDied","Data":"7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013"}
Oct 31 00:48:42.319843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.319723    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm" event={"ID":"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5","Type":"ContainerDied","Data":"e8f8cb1a7d299987e850c6558f0a1c0a47fea9b6715408f1e0c92c70c360268a"}
Oct 31 00:48:42.326248 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.326227    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"
Oct 31 00:48:42.326448 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.326248    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258" event={"ID":"fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30","Type":"ContainerDied","Data":"954e87b26071f601136ed8e987c68b337bf81e7ebd6e8629d586b52a6e59756b"}
Oct 31 00:48:42.329322 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.328794    4924 generic.go:356] "Generic (PLEG): container finished" podID="7945106c-574a-4b09-8d93-4366e9d0aa07" containerID="ee1a8df48f3d8c1e5ddec52c98e0f4032dd462b7dc9ac4d7fac99772e405d041" exitCode=0
Oct 31 00:48:42.329322 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.328854    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc" event={"ID":"7945106c-574a-4b09-8d93-4366e9d0aa07","Type":"ContainerDied","Data":"ee1a8df48f3d8c1e5ddec52c98e0f4032dd462b7dc9ac4d7fac99772e405d041"}
Oct 31 00:48:42.330416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.330271    4924 scope.go:122] "RemoveContainer" containerID="7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013"
Oct 31 00:48:42.330416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.332926    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache\") pod \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") "
Oct 31 00:48:42.330416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.332994    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm\") pod \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") "
Oct 31 00:48:42.330416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.333075    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6\" (UniqueName: \"kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6\") pod \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") "
Oct 31 00:48:42.333350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.333207    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs\") pod \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") "
Oct 31 00:48:42.333350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.333259    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache" pod "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" (UID: "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:42.333350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.333288    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home\") pod \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") "
Oct 31 00:48:42.333555 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.333378    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir\") pod \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\" (UID: \"5e2d53f7-fe8b-4eec-8648-f3949ca77ca5\") "
Oct 31 00:48:42.333606 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.333563    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home" pod "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" (UID: "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:42.334193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.334171    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:42.334279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.334195    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:42.335151 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.335126    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir" pod "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" (UID: "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:42.336251 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.336199    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm" pod "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" (UID: "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:42.336332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.336313    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6" pod "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" (UID: "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5"). InnerVolumeSpecName "kube-api-access-t2xk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:42.337173 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.337148    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs" pod "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" (UID: "5e2d53f7-fe8b-4eec-8648-f3949ca77ca5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:42.373182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.373112    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"]
Oct 31 00:48:42.382229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.380508    4924 scope.go:122] "RemoveContainer" containerID="7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013"
Oct 31 00:48:42.382229 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:42.381079    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013\": container with ID starting with 7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013 not found: ID does not exist" containerID="7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013"
Oct 31 00:48:42.382229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.381112    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013"} err="failed to get container status \"7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013\": rpc error: code = NotFound desc = could not find container \"7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013\": container with ID starting with 7000105874cab73097614cf015a188f503ccadf66eb176cea85225154d165013 not found: ID does not exist"
Oct 31 00:48:42.382229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.381129    4924 scope.go:122] "RemoveContainer" containerID="6863e752021b11433718bd0183023e98982d57b28fb8db0f7446e8c571554cfd"
Oct 31 00:48:42.383421 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.383393    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d4a7e-openshift-default-559b45695f-hl97w"]
Oct 31 00:48:42.390984 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.390931    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"]
Oct 31 00:48:42.398108 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.398089    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-44bc67-openshift-default-5b7c5d8f6c-kg258"]
Oct 31 00:48:42.435842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.435811    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:42.435842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.435839    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2xk6\" (UniqueName: \"kubernetes.io/projected/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-kube-api-access-t2xk6\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:42.436025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.435852    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:42.436025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.435864    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:42.654365 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.654327    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"]
Oct 31 00:48:42.667556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.667524    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"]
Oct 31 00:48:42.676519 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.676492    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/distinct-2-w0-f99ac0-kserve-69b4dc5898-t7mhm"]
Oct 31 00:48:42.682182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.682155    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"]
Oct 31 00:48:42.932610 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.932516    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main" containerID="cri-o://33eec5c6f708d8d486d6733b44b2dca320f7cdecde4b46a86fcfa20fe063d6ff" gracePeriod=45
Oct 31 00:48:42.958038 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:42.957992    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main" containerID="cri-o://6abc6027a9bc93a7bccbf7cf2ed5411e20eb139c7999c60e07c722926070514c" gracePeriod=45
Oct 31 00:48:43.444634 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.444111    4924 generic.go:356] "Generic (PLEG): container finished" podID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerID="33eec5c6f708d8d486d6733b44b2dca320f7cdecde4b46a86fcfa20fe063d6ff" exitCode=0
Oct 31 00:48:43.444634 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.444274    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" event={"ID":"afcac907-e8a5-4b38-9523-01f69b0a2058","Type":"ContainerDied","Data":"33eec5c6f708d8d486d6733b44b2dca320f7cdecde4b46a86fcfa20fe063d6ff"}
Oct 31 00:48:43.451570 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.450994    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:48:43.456779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.452312    4924 generic.go:356] "Generic (PLEG): container finished" podID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerID="6abc6027a9bc93a7bccbf7cf2ed5411e20eb139c7999c60e07c722926070514c" exitCode=0
Oct 31 00:48:43.456779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.452574    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" event={"ID":"b2719124-a43f-451b-b627-e4e20d8d8fd3","Type":"ContainerDied","Data":"6abc6027a9bc93a7bccbf7cf2ed5411e20eb139c7999c60e07c722926070514c"}
Oct 31 00:48:43.456779 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.452616    4924 scope.go:122] "RemoveContainer" containerID="6abc6027a9bc93a7bccbf7cf2ed5411e20eb139c7999c60e07c722926070514c"
Oct 31 00:48:43.499023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.498933    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:48:43.593212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.593172    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm\") pod \"afcac907-e8a5-4b38-9523-01f69b0a2058\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") "
Oct 31 00:48:43.593506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.593484    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home\") pod \"afcac907-e8a5-4b38-9523-01f69b0a2058\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") "
Oct 31 00:48:43.593629 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.593616    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home\") pod \"b2719124-a43f-451b-b627-e4e20d8d8fd3\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") "
Oct 31 00:48:43.593808 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.593792    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir\") pod \"afcac907-e8a5-4b38-9523-01f69b0a2058\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") "
Oct 31 00:48:43.593958 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.593943    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir\") pod \"b2719124-a43f-451b-b627-e4e20d8d8fd3\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") "
Oct 31 00:48:43.594150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594137    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs\" (UniqueName: \"kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs\") pod \"b2719124-a43f-451b-b627-e4e20d8d8fd3\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") "
Oct 31 00:48:43.594277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594262    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache\") pod \"b2719124-a43f-451b-b627-e4e20d8d8fd3\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") "
Oct 31 00:48:43.594413 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594399    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj\" (UniqueName: \"kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj\") pod \"afcac907-e8a5-4b38-9523-01f69b0a2058\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") "
Oct 31 00:48:43.594530 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594516    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4\" (UniqueName: \"kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4\") pod \"b2719124-a43f-451b-b627-e4e20d8d8fd3\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") "
Oct 31 00:48:43.594639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594626    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs\" (UniqueName: \"kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs\") pod \"afcac907-e8a5-4b38-9523-01f69b0a2058\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") "
Oct 31 00:48:43.594772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594741    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm\") pod \"b2719124-a43f-451b-b627-e4e20d8d8fd3\" (UID: \"b2719124-a43f-451b-b627-e4e20d8d8fd3\") "
Oct 31 00:48:43.594919 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.594905    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache\") pod \"afcac907-e8a5-4b38-9523-01f69b0a2058\" (UID: \"afcac907-e8a5-4b38-9523-01f69b0a2058\") "
Oct 31 00:48:43.604875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.604839    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache" pod "b2719124-a43f-451b-b627-e4e20d8d8fd3" (UID: "b2719124-a43f-451b-b627-e4e20d8d8fd3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.605268 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.605246    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home" pod "b2719124-a43f-451b-b627-e4e20d8d8fd3" (UID: "b2719124-a43f-451b-b627-e4e20d8d8fd3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.608042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.608015    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache" pod "afcac907-e8a5-4b38-9523-01f69b0a2058" (UID: "afcac907-e8a5-4b38-9523-01f69b0a2058"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.608595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.608574    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.608704 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.608692    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.618533 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.618513    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.619116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.612933    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm" pod "afcac907-e8a5-4b38-9523-01f69b0a2058" (UID: "afcac907-e8a5-4b38-9523-01f69b0a2058"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.619116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.613125    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home" pod "afcac907-e8a5-4b38-9523-01f69b0a2058" (UID: "afcac907-e8a5-4b38-9523-01f69b0a2058"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.619116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.618296    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir" pod "b2719124-a43f-451b-b627-e4e20d8d8fd3" (UID: "b2719124-a43f-451b-b627-e4e20d8d8fd3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.619116 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.618358    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir" pod "afcac907-e8a5-4b38-9523-01f69b0a2058" (UID: "afcac907-e8a5-4b38-9523-01f69b0a2058"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.619577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.619537    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm" pod "b2719124-a43f-451b-b627-e4e20d8d8fd3" (UID: "b2719124-a43f-451b-b627-e4e20d8d8fd3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:43.620209 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.620171    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj" pod "afcac907-e8a5-4b38-9523-01f69b0a2058" (UID: "afcac907-e8a5-4b38-9523-01f69b0a2058"). InnerVolumeSpecName "kube-api-access-trsmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:43.622025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.622005    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs" pod "afcac907-e8a5-4b38-9523-01f69b0a2058" (UID: "afcac907-e8a5-4b38-9523-01f69b0a2058"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:43.622320 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.622293    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4" pod "b2719124-a43f-451b-b627-e4e20d8d8fd3" (UID: "b2719124-a43f-451b-b627-e4e20d8d8fd3"). InnerVolumeSpecName "kube-api-access-tmxt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:43.632256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.631685    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs" pod "b2719124-a43f-451b-b627-e4e20d8d8fd3" (UID: "b2719124-a43f-451b-b627-e4e20d8d8fd3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721757    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2719124-a43f-451b-b627-e4e20d8d8fd3-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721801    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trsmj\" (UniqueName: \"kubernetes.io/projected/afcac907-e8a5-4b38-9523-01f69b0a2058-kube-api-access-trsmj\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721816    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmxt4\" (UniqueName: \"kubernetes.io/projected/b2719124-a43f-451b-b627-e4e20d8d8fd3-kube-api-access-tmxt4\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721830    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcac907-e8a5-4b38-9523-01f69b0a2058-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721842    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721854    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721865    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721877    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afcac907-e8a5-4b38-9523-01f69b0a2058-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.721962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.721890    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2719124-a43f-451b-b627-e4e20d8d8fd3-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.789646 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.789598    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" path="/var/lib/kubelet/pods/5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5/volumes"
Oct 31 00:48:43.790318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.790294    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" path="/var/lib/kubelet/pods/5e2d53f7-fe8b-4eec-8648-f3949ca77ca5/volumes"
Oct 31 00:48:43.791277 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.791252    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" path="/var/lib/kubelet/pods/fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30/volumes"
Oct 31 00:48:43.819185 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.819159    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:43.924147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.924113    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca\") pod \"7945106c-574a-4b09-8d93-4366e9d0aa07\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") "
Oct 31 00:48:43.924360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.924193    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs\" (UniqueName: \"kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs\") pod \"7945106c-574a-4b09-8d93-4366e9d0aa07\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") "
Oct 31 00:48:43.924360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.924323    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca\") pod \"7945106c-574a-4b09-8d93-4366e9d0aa07\" (UID: \"7945106c-574a-4b09-8d93-4366e9d0aa07\") "
Oct 31 00:48:43.924862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.924825    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca" pod "7945106c-574a-4b09-8d93-4366e9d0aa07" (UID: "7945106c-574a-4b09-8d93-4366e9d0aa07"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:43.924992 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.924972    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:43.926656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.926629    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs" pod "7945106c-574a-4b09-8d93-4366e9d0aa07" (UID: "7945106c-574a-4b09-8d93-4366e9d0aa07"). InnerVolumeSpecName "kube-api-access-zrnvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:43.926755 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:43.926692    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca" pod "7945106c-574a-4b09-8d93-4366e9d0aa07" (UID: "7945106c-574a-4b09-8d93-4366e9d0aa07"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:44.026454 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.026416    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7945106c-574a-4b09-8d93-4366e9d0aa07-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:44.026454 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.026454    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrnvs\" (UniqueName: \"kubernetes.io/projected/7945106c-574a-4b09-8d93-4366e9d0aa07-kube-api-access-zrnvs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:44.416239 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.416131    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"]
Oct 31 00:48:44.417150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417107    4924 cpu_manager.go:414] "containerMap: removing container" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy"
Oct 31 00:48:44.417150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417139    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417160    4924 cpu_manager.go:414] "containerMap: removing container" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417169    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417183    4924 cpu_manager.go:414] "containerMap: removing container" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417191    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417215    4924 cpu_manager.go:414] "containerMap: removing container" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417222    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417248    4924 cpu_manager.go:414] "containerMap: removing container" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417255    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417273    4924 cpu_manager.go:414] "containerMap: removing container" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417281    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417294    4924 cpu_manager.go:414] "containerMap: removing container" podUID="7945106c-574a-4b09-8d93-4366e9d0aa07" containerName="revoke-keys"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417302    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7945106c-574a-4b09-8d93-4366e9d0aa07" containerName="revoke-keys"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417324    4924 cpu_manager.go:414] "containerMap: removing container" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerName="payload-pre-processing"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417331    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerName="payload-pre-processing"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417355    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy"
Oct 31 00:48:44.417360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417362    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417372    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417379    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417394    4924 cpu_manager.go:414] "containerMap: removing container" podUID="0e783ee8-1151-4009-b889-de32cb744cbc" containerName="revoke-keys"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417401    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e783ee8-1151-4009-b889-de32cb744cbc" containerName="revoke-keys"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417530    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d6e0c4a-ba2a-492e-b7da-0bd5e24bb5a5" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417545    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e2d53f7-fe8b-4eec-8648-f3949ca77ca5" containerName="main"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417558    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e783ee8-1151-4009-b889-de32cb744cbc" containerName="revoke-keys"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417571    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417580    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" containerName="main"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417589    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cb74ba3-f886-4ee9-80e6-d85a149092a7" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417600    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f0ad9de-782c-41ae-87af-ea375f20c260" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417611    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" containerName="main"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417621    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd6c6f7d-da40-4c1f-8b0d-44c02cfe4d30" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417630    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417640    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="7945106c-574a-4b09-8d93-4366e9d0aa07" containerName="revoke-keys"
Oct 31 00:48:44.418137 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.417650    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="54fe85ab-8589-424f-aea2-4b5fe867ca4c" containerName="payload-pre-processing"
Oct 31 00:48:44.432101 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.432065    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"]
Oct 31 00:48:44.432275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.432208    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.498055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.498023    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"
Oct 31 00:48:44.499056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.498060    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk" event={"ID":"b2719124-a43f-451b-b627-e4e20d8d8fd3","Type":"ContainerDied","Data":"542f7e89e02a2b28ef3d5cf5be67f8fa2a4746d1cb17f5a005480f6f115e365e"}
Oct 31 00:48:44.500142 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.500120    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"
Oct 31 00:48:44.500320 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.500123    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b" event={"ID":"afcac907-e8a5-4b38-9523-01f69b0a2058","Type":"ContainerDied","Data":"80ffea2bfad3da51c0e3f2446e76ae699b7a16406ee9b3893d989fd5b1929110"}
Oct 31 00:48:44.500320 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.500268    4924 scope.go:122] "RemoveContainer" containerID="33eec5c6f708d8d486d6733b44b2dca320f7cdecde4b46a86fcfa20fe063d6ff"
Oct 31 00:48:44.502840 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.502822    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"
Oct 31 00:48:44.503028 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.502862    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc" event={"ID":"7945106c-574a-4b09-8d93-4366e9d0aa07","Type":"ContainerDied","Data":"05fb53da59066fd30fd9bd6251de77c9e56d184a9aaa2598558db4c4c795baa0"}
Oct 31 00:48:44.503322 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.503306    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05fb53da59066fd30fd9bd6251de77c9e56d184a9aaa2598558db4c4c795baa0"
Oct 31 00:48:44.541581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.540209    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkf5\" (UniqueName: \"kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.541581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.540604    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.541581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.540822    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.546881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.546853    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"]
Oct 31 00:48:44.553100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.553073    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/premium-w0-f99ac0-kserve-7db85fb86-wtr5b"]
Oct 31 00:48:44.566897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.566871    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"]
Oct 31 00:48:44.574128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.574090    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f99ac0/simulator-w0-f99ac0-kserve-594dc4bcf-wswbk"]
Oct 31 00:48:44.642030 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.641938    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.642030 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.642027    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkf5\" (UniqueName: \"kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.642272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.642143    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.642713 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.642691    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.644634 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.644608    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.650416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.650387    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkf5\" (UniqueName: \"kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5\") pod \"maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.752612 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.752571    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:44.972894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:44.972799    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" podUID="e9f27690-bb8e-4159-b918-84baff256f09" containerName="payload-processing" containerID="cri-o://486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7" gracePeriod=25
Oct 31 00:48:45.047587 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:45.047555    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5e3e23_760a_44d5_a4b3_5806dadefcab.slice/crio-ea1b5480eaeccddc5a6ff281fc7ec19cc73ee7197241e3fc98f547fea65d11fd WatchSource:0}: Error finding container ea1b5480eaeccddc5a6ff281fc7ec19cc73ee7197241e3fc98f547fea65d11fd: Status 404 returned error can't find the container with id ea1b5480eaeccddc5a6ff281fc7ec19cc73ee7197241e3fc98f547fea65d11fd
Oct 31 00:48:45.049035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:45.049006    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"]
Oct 31 00:48:45.515917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:45.515871    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw" event={"ID":"da5e3e23-760a-44d5-a4b3-5806dadefcab","Type":"ContainerStarted","Data":"67a1ce7a4d5c000a8ef085cfd5f76b793c027211997b12c7a922a482219d5b48"}
Oct 31 00:48:45.548285 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:45.516890    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw" event={"ID":"da5e3e23-760a-44d5-a4b3-5806dadefcab","Type":"ContainerStarted","Data":"ea1b5480eaeccddc5a6ff281fc7ec19cc73ee7197241e3fc98f547fea65d11fd"}
Oct 31 00:48:45.548285 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:45.543960    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw" podStartSLOduration=1.543945034 podStartE2EDuration="1.543945034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:45.543726193 +0000 UTC m=+925.914509985" watchObservedRunningTime="2026-10-31 00:48:45.543945034 +0000 UTC m=+925.914728823"
Oct 31 00:48:45.765960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:45.765926    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afcac907-e8a5-4b38-9523-01f69b0a2058" path="/var/lib/kubelet/pods/afcac907-e8a5-4b38-9523-01f69b0a2058/volumes"
Oct 31 00:48:45.766512 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:45.766493    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2719124-a43f-451b-b627-e4e20d8d8fd3" path="/var/lib/kubelet/pods/b2719124-a43f-451b-b627-e4e20d8d8fd3/volumes"
Oct 31 00:48:46.530055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:46.530020    4924 generic.go:356] "Generic (PLEG): container finished" podID="da5e3e23-760a-44d5-a4b3-5806dadefcab" containerID="67a1ce7a4d5c000a8ef085cfd5f76b793c027211997b12c7a922a482219d5b48" exitCode=0
Oct 31 00:48:46.530538 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:46.530108    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw" event={"ID":"da5e3e23-760a-44d5-a4b3-5806dadefcab","Type":"ContainerDied","Data":"67a1ce7a4d5c000a8ef085cfd5f76b793c027211997b12c7a922a482219d5b48"}
Oct 31 00:48:47.741945 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.741911    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:47.782391 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.782355    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca\") pod \"da5e3e23-760a-44d5-a4b3-5806dadefcab\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") "
Oct 31 00:48:47.782573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.782551    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5\" (UniqueName: \"kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5\") pod \"da5e3e23-760a-44d5-a4b3-5806dadefcab\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") "
Oct 31 00:48:47.782715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.782587    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca\") pod \"da5e3e23-760a-44d5-a4b3-5806dadefcab\" (UID: \"da5e3e23-760a-44d5-a4b3-5806dadefcab\") "
Oct 31 00:48:47.783179 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.783146    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca" pod "da5e3e23-760a-44d5-a4b3-5806dadefcab" (UID: "da5e3e23-760a-44d5-a4b3-5806dadefcab"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:47.783636 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.783598    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:47.784834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.784809    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca" pod "da5e3e23-760a-44d5-a4b3-5806dadefcab" (UID: "da5e3e23-760a-44d5-a4b3-5806dadefcab"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:47.785598 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.785571    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5" pod "da5e3e23-760a-44d5-a4b3-5806dadefcab" (UID: "da5e3e23-760a-44d5-a4b3-5806dadefcab"). InnerVolumeSpecName "kube-api-access-7xkf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:47.885767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.885605    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xkf5\" (UniqueName: \"kubernetes.io/projected/da5e3e23-760a-44d5-a4b3-5806dadefcab-kube-api-access-7xkf5\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:47.885767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:47.885649    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/da5e3e23-760a-44d5-a4b3-5806dadefcab-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:48.131890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.131794    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"]
Oct 31 00:48:48.132087 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.132058    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" containerID="cri-o://fe7a23eef0a729873c1c34ef15a348b8c0440f8359827c5df857081028a71ba5" gracePeriod=30
Oct 31 00:48:48.140681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.140640    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"]
Oct 31 00:48:48.140943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.140912    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy" containerID="cri-o://2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10" gracePeriod=30
Oct 31 00:48:48.151165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.150586    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"]
Oct 31 00:48:48.151165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.150844    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy" containerID="cri-o://fdbe9b9d116b9a5bd6e71fdf728dce9db4be3bc5ea9e1889b768296c454d7da2" gracePeriod=30
Oct 31 00:48:48.552416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.552378    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw" event={"ID":"da5e3e23-760a-44d5-a4b3-5806dadefcab","Type":"ContainerDied","Data":"ea1b5480eaeccddc5a6ff281fc7ec19cc73ee7197241e3fc98f547fea65d11fd"}
Oct 31 00:48:48.552416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.552416    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"
Oct 31 00:48:48.552688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:48.552429    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea1b5480eaeccddc5a6ff281fc7ec19cc73ee7197241e3fc98f547fea65d11fd"
Oct 31 00:48:49.122397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:49.122352    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"]
Oct 31 00:48:50.054761 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.054704    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"]
Oct 31 00:48:50.055665 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.055641    4924 cpu_manager.go:414] "containerMap: removing container" podUID="da5e3e23-760a-44d5-a4b3-5806dadefcab" containerName="revoke-keys"
Oct 31 00:48:50.055665 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.055662    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da5e3e23-760a-44d5-a4b3-5806dadefcab" containerName="revoke-keys"
Oct 31 00:48:50.055864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.055688    4924 cpu_manager.go:414] "containerMap: removing container" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy"
Oct 31 00:48:50.055864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.055698    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afdebed2-b2b3-4856-9af3-13a739db3fdc" containerName="istio-proxy"
Oct 31 00:48:50.055864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.055838    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="da5e3e23-760a-44d5-a4b3-5806dadefcab" containerName="revoke-keys"
Oct 31 00:48:50.060158 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.060138    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.061066 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.061038    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"]
Oct 31 00:48:50.065577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.065191    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:48:50.066153 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.066129    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.070891 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.070865    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"]
Oct 31 00:48:50.071449 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.071427    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:48:50.113207 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.109569    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"]
Oct 31 00:48:50.142189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.142150    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.142591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.142306    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.142591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.142369    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.142591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.142422    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-256zf\" (UniqueName: \"kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.142591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.142541    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf2hl\" (UniqueName: \"kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.142591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.142567    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.244311 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.244259    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf2hl\" (UniqueName: \"kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.244513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.244426    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.244513 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.244500    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.244688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.244667    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.244786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.244703    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.244862 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.244812    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-256zf\" (UniqueName: \"kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.245305 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.245273    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.245436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.245314    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.247122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.247096    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.247217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.247140    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.253536 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.253510    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-256zf\" (UniqueName: \"kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf\") pod \"maas-api-revoke-sub-faed4c37-8wd46\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.253633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.253606    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf2hl\" (UniqueName: \"kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl\") pod \"maas-api-revoke-sub-dabf9fd9-dgtsx\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.402827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.402549    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:50.406771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.403141    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:50.729713 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:50.729674    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e35c113_5708_417e_988e_537f2c3d8558.slice/crio-15d7e2b39abddf3b887bb6386f5373ecc3d252fc58d795f3fb9146bfc88022fd WatchSource:0}: Error finding container 15d7e2b39abddf3b887bb6386f5373ecc3d252fc58d795f3fb9146bfc88022fd: Status 404 returned error can't find the container with id 15d7e2b39abddf3b887bb6386f5373ecc3d252fc58d795f3fb9146bfc88022fd
Oct 31 00:48:50.732002 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.731981    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"]
Oct 31 00:48:50.734109 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:50.734078    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a2c7e2_3b51_4c2f_bfce_0ba7183808a1.slice/crio-8e5f735ffa6a7f963251c087c4fb3c9b44c39a05dade6bd82d1e11fa607478b2 WatchSource:0}: Error finding container 8e5f735ffa6a7f963251c087c4fb3c9b44c39a05dade6bd82d1e11fa607478b2: Status 404 returned error can't find the container with id 8e5f735ffa6a7f963251c087c4fb3c9b44c39a05dade6bd82d1e11fa607478b2
Oct 31 00:48:50.740611 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:50.740568    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"]
Oct 31 00:48:51.166412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.166375    4924 patch_prober.go:29] interesting pod/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:48:51.166953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.166432    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:48:51.593146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.593104    4924 generic.go:356] "Generic (PLEG): container finished" podID="5e35c113-5708-417e-988e-537f2c3d8558" containerID="a349585f0d067514993c5f00ad98e90ceb1032517f4ed25a2d0562374aaf9bac" exitCode=0
Oct 31 00:48:51.593345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.593198    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46" event={"ID":"5e35c113-5708-417e-988e-537f2c3d8558","Type":"ContainerDied","Data":"a349585f0d067514993c5f00ad98e90ceb1032517f4ed25a2d0562374aaf9bac"}
Oct 31 00:48:51.593345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.593273    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46" event={"ID":"5e35c113-5708-417e-988e-537f2c3d8558","Type":"ContainerStarted","Data":"15d7e2b39abddf3b887bb6386f5373ecc3d252fc58d795f3fb9146bfc88022fd"}
Oct 31 00:48:51.595855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.595812    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx" event={"ID":"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1","Type":"ContainerStarted","Data":"5f85453d10876b497dce09427167cb40a674398ce7e1d2f195bbcae15d9c3863"}
Oct 31 00:48:51.596007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.595868    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx" event={"ID":"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1","Type":"ContainerStarted","Data":"8e5f735ffa6a7f963251c087c4fb3c9b44c39a05dade6bd82d1e11fa607478b2"}
Oct 31 00:48:51.659345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:51.659274    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx" podStartSLOduration=1.659254113 podStartE2EDuration="1.659254113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:51.655706284 +0000 UTC m=+932.026490068" watchObservedRunningTime="2026-10-31 00:48:51.659254113 +0000 UTC m=+932.030037904"
Oct 31 00:48:52.283833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.283788    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" podUID="e9f27690-bb8e-4159-b918-84baff256f09" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:48:52.633836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.620172    4924 generic.go:356] "Generic (PLEG): container finished" podID="f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" containerID="5f85453d10876b497dce09427167cb40a674398ce7e1d2f195bbcae15d9c3863" exitCode=0
Oct 31 00:48:52.633836 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.620306    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx" event={"ID":"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1","Type":"ContainerDied","Data":"5f85453d10876b497dce09427167cb40a674398ce7e1d2f195bbcae15d9c3863"}
Oct 31 00:48:52.836860 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.836826    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:52.882601 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.882563    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca\") pod \"5e35c113-5708-417e-988e-537f2c3d8558\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") "
Oct 31 00:48:52.882839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.882675    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf\" (UniqueName: \"kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf\") pod \"5e35c113-5708-417e-988e-537f2c3d8558\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") "
Oct 31 00:48:52.882839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.882813    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca\") pod \"5e35c113-5708-417e-988e-537f2c3d8558\" (UID: \"5e35c113-5708-417e-988e-537f2c3d8558\") "
Oct 31 00:48:52.883194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.883164    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca" pod "5e35c113-5708-417e-988e-537f2c3d8558" (UID: "5e35c113-5708-417e-988e-537f2c3d8558"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:52.883377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.883358    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:52.885031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.884975    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca" pod "5e35c113-5708-417e-988e-537f2c3d8558" (UID: "5e35c113-5708-417e-988e-537f2c3d8558"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:52.885031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.884975    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf" pod "5e35c113-5708-417e-988e-537f2c3d8558" (UID: "5e35c113-5708-417e-988e-537f2c3d8558"). InnerVolumeSpecName "kube-api-access-256zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:52.985186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.985140    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-256zf\" (UniqueName: \"kubernetes.io/projected/5e35c113-5708-417e-988e-537f2c3d8558-kube-api-access-256zf\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:52.985186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:52.985179    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5e35c113-5708-417e-988e-537f2c3d8558-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.607019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.606981    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:48:53.638783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.638630    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46" event={"ID":"5e35c113-5708-417e-988e-537f2c3d8558","Type":"ContainerDied","Data":"15d7e2b39abddf3b887bb6386f5373ecc3d252fc58d795f3fb9146bfc88022fd"}
Oct 31 00:48:53.638783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.638683    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15d7e2b39abddf3b887bb6386f5373ecc3d252fc58d795f3fb9146bfc88022fd"
Oct 31 00:48:53.638783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.638734    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"
Oct 31 00:48:53.640887 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.640852    4924 generic.go:356] "Generic (PLEG): container finished" podID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerID="2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10" exitCode=0
Oct 31 00:48:53.641240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.641218    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" event={"ID":"aa1eab11-d772-43c7-94ac-8a58d445ce90","Type":"ContainerDied","Data":"2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10"}
Oct 31 00:48:53.641383 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.641368    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg" event={"ID":"aa1eab11-d772-43c7-94ac-8a58d445ce90","Type":"ContainerDied","Data":"51a6ee5d27f0c005366ffc5a0e210793b92b93cbc89696a515f4bd107f89c4df"}
Oct 31 00:48:53.641534 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.641521    4924 scope.go:122] "RemoveContainer" containerID="2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10"
Oct 31 00:48:53.641829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.641806    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"
Oct 31 00:48:53.645146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.645118    4924 generic.go:356] "Generic (PLEG): container finished" podID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerID="fe7a23eef0a729873c1c34ef15a348b8c0440f8359827c5df857081028a71ba5" exitCode=0
Oct 31 00:48:53.645436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.645415    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" event={"ID":"c13c58cb-6288-4c94-82b9-8ef44e90c10d","Type":"ContainerDied","Data":"fe7a23eef0a729873c1c34ef15a348b8c0440f8359827c5df857081028a71ba5"}
Oct 31 00:48:53.647730 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.647704    4924 generic.go:356] "Generic (PLEG): container finished" podID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerID="fdbe9b9d116b9a5bd6e71fdf728dce9db4be3bc5ea9e1889b768296c454d7da2" exitCode=0
Oct 31 00:48:53.649962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.648204    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" event={"ID":"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02","Type":"ContainerDied","Data":"fdbe9b9d116b9a5bd6e71fdf728dce9db4be3bc5ea9e1889b768296c454d7da2"}
Oct 31 00:48:53.674054 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.674007    4924 scope.go:122] "RemoveContainer" containerID="2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10"
Oct 31 00:48:53.674693 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:53.674621    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10\": container with ID starting with 2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10 not found: ID does not exist" containerID="2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10"
Oct 31 00:48:53.674693 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.674661    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10"} err="failed to get container status \"2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10\": rpc error: code = NotFound desc = could not find container \"2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10\": container with ID starting with 2131e08fba06527a7c8860b3fe674b38aebd0a9333624422c1376ea52203aa10 not found: ID does not exist"
Oct 31 00:48:53.699385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.698917    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.699385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.699054    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.699385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.699088    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.699385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.699149    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.699385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.699201    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.699385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.699263    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.701127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.699912    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.701127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.700026    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.701127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.700052    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket\") pod \"aa1eab11-d772-43c7-94ac-8a58d445ce90\" (UID: \"aa1eab11-d772-43c7-94ac-8a58d445ce90\") "
Oct 31 00:48:53.701127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.700195    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.701127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.700689    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.701127 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.701093    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.701445 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.701122    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.701489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.701469    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:53.701590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.701562    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.702660 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.702638    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:53.704911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.704883    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:53.705001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.704975    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.706313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.705714    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4" pod "aa1eab11-d772-43c7-94ac-8a58d445ce90" (UID: "aa1eab11-d772-43c7-94ac-8a58d445ce90"). InnerVolumeSpecName "kube-api-access-gnpf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:53.708453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.708432    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:48:53.731942 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.731911    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:48:53.801959 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.801929    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.802581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.802558    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.803057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.803038    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.803731 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.803712    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.804449 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.804429    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.805050 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.805032    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.805431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.805415    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.805894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.805877    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.806000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.802962    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.806058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.804335    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:53.806058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.804915    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.806058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.805355    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.806314 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.806242    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.806648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.806632    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.807122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.807101    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.807532 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.807516    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.808045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.808028    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.808490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.808469    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.808875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.808859    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.809335 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.809265    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token\") pod \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\" (UID: \"c13c58cb-6288-4c94-82b9-8ef44e90c10d\") "
Oct 31 00:48:53.809783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.809728    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.809935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.809917    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data\") pod \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\" (UID: \"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02\") "
Oct 31 00:48:53.810850 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.810831    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.810977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.810962    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811052    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811136 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811126    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811192    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811306    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811373    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811467 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811457    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.811539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.811529    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.813055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.813035    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa1eab11-d772-43c7-94ac-8a58d445ce90-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.813482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.813467    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa1eab11-d772-43c7-94ac-8a58d445ce90-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.813657 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.813642    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnpf4\" (UniqueName: \"kubernetes.io/projected/aa1eab11-d772-43c7-94ac-8a58d445ce90-kube-api-access-gnpf4\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.814104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.806430    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:53.814206 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.806577    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.814274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.807452    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.814355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.808347    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.814433 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.808790    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.819431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.819379    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.820617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.819427    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:53.820617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.819647    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.821196 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.821153    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:53.824051 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.824012    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:48:53.826148 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.826126    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:53.826284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.826146    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "kube-api-access-bqg8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:53.827070 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.827042    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5" pod "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" (UID: "c8be0b53-3a6f-4b9f-8cb8-4c24179eca02"). InnerVolumeSpecName "kube-api-access-6blg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:53.838165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.838129    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy" pod "c13c58cb-6288-4c94-82b9-8ef44e90c10d" (UID: "c13c58cb-6288-4c94-82b9-8ef44e90c10d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:53.888507 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.888471    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:53.915577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915542    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6blg5\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-kube-api-access-6blg5\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915575    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqg8x\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-kube-api-access-bqg8x\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915586    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915596    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915608    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915621    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915629    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915638    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915646    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13c58cb-6288-4c94-82b9-8ef44e90c10d-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915656    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915666    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915674    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13c58cb-6288-4c94-82b9-8ef44e90c10d-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915687    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.915914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.915697    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:53.977466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.977370    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"]
Oct 31 00:48:53.983098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:53.983060    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-r8ksg"]
Oct 31 00:48:54.017042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.017004    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl\" (UniqueName: \"kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl\") pod \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") "
Oct 31 00:48:54.017263 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.017134    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca\") pod \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") "
Oct 31 00:48:54.017263 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.017233    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca\") pod \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\" (UID: \"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1\") "
Oct 31 00:48:54.017616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.017592    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca" pod "f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" (UID: "f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:54.017757 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.017732    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.019340 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.019317    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca" pod "f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" (UID: "f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:54.019423 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.019385    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl" pod "f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" (UID: "f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1"). InnerVolumeSpecName "kube-api-access-cf2hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:54.119044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.119006    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.119044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.119038    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf2hl\" (UniqueName: \"kubernetes.io/projected/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1-kube-api-access-cf2hl\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.655367 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.655327    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"]
Oct 31 00:48:54.662403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.662370    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"
Oct 31 00:48:54.662403 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.662384    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx" event={"ID":"f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1","Type":"ContainerDied","Data":"8e5f735ffa6a7f963251c087c4fb3c9b44c39a05dade6bd82d1e11fa607478b2"}
Oct 31 00:48:54.662675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.662442    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e5f735ffa6a7f963251c087c4fb3c9b44c39a05dade6bd82d1e11fa607478b2"
Oct 31 00:48:54.668688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.668187    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9" event={"ID":"c13c58cb-6288-4c94-82b9-8ef44e90c10d","Type":"ContainerDied","Data":"7b915bda35d011a7a908ee7c95cab512c973e15be4ec7285ad852a5fd276adb1"}
Oct 31 00:48:54.668688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.668262    4924 scope.go:122] "RemoveContainer" containerID="fe7a23eef0a729873c1c34ef15a348b8c0440f8359827c5df857081028a71ba5"
Oct 31 00:48:54.668688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.668389    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"
Oct 31 00:48:54.677431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.675639    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c" event={"ID":"c8be0b53-3a6f-4b9f-8cb8-4c24179eca02","Type":"ContainerDied","Data":"5423f79a252579fbad69caed3b97ba5cc8a9a2ca9f4660c87d10d79718b75928"}
Oct 31 00:48:54.677431 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.675778    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"
Oct 31 00:48:54.698556 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.697483    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"]
Oct 31 00:48:54.698848 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.698822    4924 scope.go:122] "RemoveContainer" containerID="fdbe9b9d116b9a5bd6e71fdf728dce9db4be3bc5ea9e1889b768296c454d7da2"
Oct 31 00:48:54.732625 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.732587    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"]
Oct 31 00:48:54.750827 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.750793    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-9ffg9"]
Oct 31 00:48:54.804834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.804796    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"]
Oct 31 00:48:54.809892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:54.809862    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f1496-openshift-default-8665bf5969-7bh4c"]
Oct 31 00:48:55.765959 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:55.765919    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" path="/var/lib/kubelet/pods/aa1eab11-d772-43c7-94ac-8a58d445ce90/volumes"
Oct 31 00:48:55.766504 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:55.766486    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" path="/var/lib/kubelet/pods/c13c58cb-6288-4c94-82b9-8ef44e90c10d/volumes"
Oct 31 00:48:55.766971 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:55.766956    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" path="/var/lib/kubelet/pods/c8be0b53-3a6f-4b9f-8cb8-4c24179eca02/volumes"
Oct 31 00:48:56.128875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.128786    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"]
Oct 31 00:48:56.129351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129337    4924 cpu_manager.go:414] "containerMap: removing container" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129351    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129366    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129371    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129387    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5e35c113-5708-417e-988e-537f2c3d8558" containerName="revoke-keys"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129392    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e35c113-5708-417e-988e-537f2c3d8558" containerName="revoke-keys"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129405    4924 cpu_manager.go:414] "containerMap: removing container" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy"
Oct 31 00:48:56.129408 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129410    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129431    4924 cpu_manager.go:414] "containerMap: removing container" podUID="f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" containerName="revoke-keys"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129435    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" containerName="revoke-keys"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129525    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" containerName="revoke-keys"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129536    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa1eab11-d772-43c7-94ac-8a58d445ce90" containerName="istio-proxy"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129543    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="c13c58cb-6288-4c94-82b9-8ef44e90c10d" containerName="istio-proxy"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129551    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e35c113-5708-417e-988e-537f2c3d8558" containerName="revoke-keys"
Oct 31 00:48:56.129692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.129558    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8be0b53-3a6f-4b9f-8cb8-4c24179eca02" containerName="istio-proxy"
Oct 31 00:48:56.135374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.135342    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.139867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.139839    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:48:56.140052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.139888    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:48:56.144726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.144630    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"]
Oct 31 00:48:56.263150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.263102    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.263313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.263171    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkj6\" (UniqueName: \"kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.263313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.263281    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.365192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.365152    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.365192 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.365199    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkj6\" (UniqueName: \"kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.365434 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.365244    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.366067 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.366038    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.367777 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.367716    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.373678 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.373648    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkj6\" (UniqueName: \"kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6\") pod \"maas-api-revoke-sub-1657762c-6sk4x\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.461923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.461834    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:56.777511 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:48:56.777460    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18a7547_2609_4876_895d_d378d5582f32.slice/crio-de34bb5cd82cafe74638c0dba95de966af1dad9e7d48a21401c3a3a8e91863cf WatchSource:0}: Error finding container de34bb5cd82cafe74638c0dba95de966af1dad9e7d48a21401c3a3a8e91863cf: Status 404 returned error can't find the container with id de34bb5cd82cafe74638c0dba95de966af1dad9e7d48a21401c3a3a8e91863cf
Oct 31 00:48:56.779380 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:56.779315    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"]
Oct 31 00:48:57.680460 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:57.680400    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main" containerID="cri-o://f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4" gracePeriod=45
Oct 31 00:48:57.704844 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:57.704808    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main" containerID="cri-o://cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389" gracePeriod=45
Oct 31 00:48:57.715679 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:57.715647    4924 generic.go:356] "Generic (PLEG): container finished" podID="a18a7547-2609-4876-895d-d378d5582f32" containerID="1e8a546c76fb0388c99a2a897d816a6002191c3840a5ec649438513beed4d05c" exitCode=0
Oct 31 00:48:57.715829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:57.715735    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x" event={"ID":"a18a7547-2609-4876-895d-d378d5582f32","Type":"ContainerDied","Data":"1e8a546c76fb0388c99a2a897d816a6002191c3840a5ec649438513beed4d05c"}
Oct 31 00:48:57.715829 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:57.715808    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x" event={"ID":"a18a7547-2609-4876-895d-d378d5582f32","Type":"ContainerStarted","Data":"de34bb5cd82cafe74638c0dba95de966af1dad9e7d48a21401c3a3a8e91863cf"}
Oct 31 00:48:57.863840 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:57.863712    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:48:58.006476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.006447    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:48:58.010735 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.010701    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090520    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home\") pod \"ac16d450-266c-4cce-a250-0725443133c4\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090585    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm\") pod \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090626    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home\") pod \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090700    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs\" (UniqueName: \"kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs\") pod \"ac16d450-266c-4cce-a250-0725443133c4\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090737    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir\") pod \"ac16d450-266c-4cce-a250-0725443133c4\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090831    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v\" (UniqueName: \"kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v\") pod \"ac16d450-266c-4cce-a250-0725443133c4\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090888    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache\") pod \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.090963    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir\") pod \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.091026    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache\") pod \"ac16d450-266c-4cce-a250-0725443133c4\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.091074    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5\" (UniqueName: \"kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5\") pod \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.091152    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm\") pod \"ac16d450-266c-4cce-a250-0725443133c4\" (UID: \"ac16d450-266c-4cce-a250-0725443133c4\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.091191    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home" pod "88b08800-ec7f-4d04-85a2-ef12a01a4c10" (UID: "88b08800-ec7f-4d04-85a2-ef12a01a4c10"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.091244    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs\" (UniqueName: \"kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs\") pod \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\" (UID: \"88b08800-ec7f-4d04-85a2-ef12a01a4c10\") "
Oct 31 00:48:58.091782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.091402    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home" pod "ac16d450-266c-4cce-a250-0725443133c4" (UID: "ac16d450-266c-4cce-a250-0725443133c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.101221 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.101185    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.101221 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.101223    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.105470 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.105408    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache" pod "88b08800-ec7f-4d04-85a2-ef12a01a4c10" (UID: "88b08800-ec7f-4d04-85a2-ef12a01a4c10"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.108594 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.108401    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir" pod "88b08800-ec7f-4d04-85a2-ef12a01a4c10" (UID: "88b08800-ec7f-4d04-85a2-ef12a01a4c10"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.108867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.108827    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache" pod "ac16d450-266c-4cce-a250-0725443133c4" (UID: "ac16d450-266c-4cce-a250-0725443133c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.109031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.108923    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir" pod "ac16d450-266c-4cce-a250-0725443133c4" (UID: "ac16d450-266c-4cce-a250-0725443133c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.111442 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.111198    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs" pod "88b08800-ec7f-4d04-85a2-ef12a01a4c10" (UID: "88b08800-ec7f-4d04-85a2-ef12a01a4c10"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:58.112089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.112057    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm" pod "88b08800-ec7f-4d04-85a2-ef12a01a4c10" (UID: "88b08800-ec7f-4d04-85a2-ef12a01a4c10"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.112285 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.112062    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs" pod "ac16d450-266c-4cce-a250-0725443133c4" (UID: "ac16d450-266c-4cce-a250-0725443133c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:58.112420 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.112388    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v" pod "ac16d450-266c-4cce-a250-0725443133c4" (UID: "ac16d450-266c-4cce-a250-0725443133c4"). InnerVolumeSpecName "kube-api-access-fsg6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:58.112781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.112693    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5" pod "88b08800-ec7f-4d04-85a2-ef12a01a4c10" (UID: "88b08800-ec7f-4d04-85a2-ef12a01a4c10"). InnerVolumeSpecName "kube-api-access-hrgc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:58.114206 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.114137    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm" pod "ac16d450-266c-4cce-a250-0725443133c4" (UID: "ac16d450-266c-4cce-a250-0725443133c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:48:58.202377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202332    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsg6v\" (UniqueName: \"kubernetes.io/projected/ac16d450-266c-4cce-a250-0725443133c4-kube-api-access-fsg6v\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202377 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202379    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202396    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202411    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202428    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrgc5\" (UniqueName: \"kubernetes.io/projected/88b08800-ec7f-4d04-85a2-ef12a01a4c10-kube-api-access-hrgc5\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202442    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202456    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b08800-ec7f-4d04-85a2-ef12a01a4c10-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202470    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88b08800-ec7f-4d04-85a2-ef12a01a4c10-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202502    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac16d450-266c-4cce-a250-0725443133c4-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.202562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.202515    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac16d450-266c-4cce-a250-0725443133c4-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:58.728007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.727901    4924 generic.go:356] "Generic (PLEG): container finished" podID="ac16d450-266c-4cce-a250-0725443133c4" containerID="cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389" exitCode=0
Oct 31 00:48:58.728007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.727974    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"
Oct 31 00:48:58.728007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.727989    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" event={"ID":"ac16d450-266c-4cce-a250-0725443133c4","Type":"ContainerDied","Data":"cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389"}
Oct 31 00:48:58.728296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.728049    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr" event={"ID":"ac16d450-266c-4cce-a250-0725443133c4","Type":"ContainerDied","Data":"8272c878f6e309d4c46cc8567834684062754aedd9973ecfa9198db4b48b1361"}
Oct 31 00:48:58.728296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.728072    4924 scope.go:122] "RemoveContainer" containerID="cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389"
Oct 31 00:48:58.730436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.730278    4924 generic.go:356] "Generic (PLEG): container finished" podID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerID="f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4" exitCode=0
Oct 31 00:48:58.730436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.730374    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"
Oct 31 00:48:58.730436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.730401    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" event={"ID":"88b08800-ec7f-4d04-85a2-ef12a01a4c10","Type":"ContainerDied","Data":"f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4"}
Oct 31 00:48:58.730698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.730447    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5" event={"ID":"88b08800-ec7f-4d04-85a2-ef12a01a4c10","Type":"ContainerDied","Data":"4bb5697ecfd04dfa004a174c19ca5b75b0049615137e94e7442eb250e480f216"}
Oct 31 00:48:58.748502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.748475    4924 scope.go:122] "RemoveContainer" containerID="cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389"
Oct 31 00:48:58.748968 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:58.748942    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389\": container with ID starting with cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389 not found: ID does not exist" containerID="cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389"
Oct 31 00:48:58.749074 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.748976    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389"} err="failed to get container status \"cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389\": rpc error: code = NotFound desc = could not find container \"cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389\": container with ID starting with cfa85a10d7253e75164cecb5921ead810119e053ef7612fa03eeec9c852bb389 not found: ID does not exist"
Oct 31 00:48:58.749074 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.748994    4924 scope.go:122] "RemoveContainer" containerID="f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4"
Oct 31 00:48:58.764707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.764675    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"]
Oct 31 00:48:58.766604 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.766491    4924 scope.go:122] "RemoveContainer" containerID="f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4"
Oct 31 00:48:58.767007 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:48:58.766985    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4\": container with ID starting with f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4 not found: ID does not exist" containerID="f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4"
Oct 31 00:48:58.767067 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.767016    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4"} err="failed to get container status \"f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4\": rpc error: code = NotFound desc = could not find container \"f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4\": container with ID starting with f0bae3c7f25c449c736183024561f0b5c381cdaafe298a2f1ef9d283cfa899f4 not found: ID does not exist"
Oct 31 00:48:58.770237 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.770210    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/simulator-w4-5c2a03-kserve-5c5d6b98b7-n9nrr"]
Oct 31 00:48:58.785440 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.785113    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"]
Oct 31 00:48:58.790672 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.790637    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-5c2a03/distinct-w4-5c2a03-kserve-ff87b69d5-rjsg5"]
Oct 31 00:48:58.925068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:58.924343    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:59.014069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.013972    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca\") pod \"a18a7547-2609-4876-895d-d378d5582f32\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") "
Oct 31 00:48:59.014069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.014036    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca\") pod \"a18a7547-2609-4876-895d-d378d5582f32\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") "
Oct 31 00:48:59.014297 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.014236    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6\" (UniqueName: \"kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6\") pod \"a18a7547-2609-4876-895d-d378d5582f32\" (UID: \"a18a7547-2609-4876-895d-d378d5582f32\") "
Oct 31 00:48:59.014557 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.014531    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca" pod "a18a7547-2609-4876-895d-d378d5582f32" (UID: "a18a7547-2609-4876-895d-d378d5582f32"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:59.014932 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.014895    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a18a7547-2609-4876-895d-d378d5582f32-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:59.017105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.017072    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca" pod "a18a7547-2609-4876-895d-d378d5582f32" (UID: "a18a7547-2609-4876-895d-d378d5582f32"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:48:59.017105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.017098    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6" pod "a18a7547-2609-4876-895d-d378d5582f32" (UID: "a18a7547-2609-4876-895d-d378d5582f32"). InnerVolumeSpecName "kube-api-access-nkkj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:59.115710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.115664    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a18a7547-2609-4876-895d-d378d5582f32-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:59.115710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.115702    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkkj6\" (UniqueName: \"kubernetes.io/projected/a18a7547-2609-4876-895d-d378d5582f32-kube-api-access-nkkj6\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:59.655755 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.655710    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerName="payload-processing" containerID="cri-o://4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb" gracePeriod=25
Oct 31 00:48:59.699313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.699260    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerName="payload-pre-processing" containerID="cri-o://59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121" gracePeriod=25
Oct 31 00:48:59.744635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.744604    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"
Oct 31 00:48:59.744635 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.744625    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x" event={"ID":"a18a7547-2609-4876-895d-d378d5582f32","Type":"ContainerDied","Data":"de34bb5cd82cafe74638c0dba95de966af1dad9e7d48a21401c3a3a8e91863cf"}
Oct 31 00:48:59.744886 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.744671    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de34bb5cd82cafe74638c0dba95de966af1dad9e7d48a21401c3a3a8e91863cf"
Oct 31 00:48:59.764912 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.764882    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" path="/var/lib/kubelet/pods/88b08800-ec7f-4d04-85a2-ef12a01a4c10/volumes"
Oct 31 00:48:59.765432 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:48:59.765417    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac16d450-266c-4cce-a250-0725443133c4" path="/var/lib/kubelet/pods/ac16d450-266c-4cce-a250-0725443133c4/volumes"
Oct 31 00:49:00.557389 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.557349    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"]
Oct 31 00:49:00.557923 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.557641    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" containerID="cri-o://4f3bd7c408cc401d8478de0c7fbfc58780e1e3b87b2a7243f08d5e82bd54f2f5" gracePeriod=30
Oct 31 00:49:00.567359 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.567330    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"]
Oct 31 00:49:00.567571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.567532    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" containerID="cri-o://7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea" gracePeriod=30
Oct 31 00:49:00.572638 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.572610    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"]
Oct 31 00:49:00.572928 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.572902    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy" containerID="cri-o://75ee4f1be45a1682a8ea28bd5106b5756b48dd12eb457bf4eb3bb2d1f61fb70a" gracePeriod=30
Oct 31 00:49:00.577893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.577865    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"]
Oct 31 00:49:00.578117 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:00.578076    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" containerID="cri-o://b0c693e04f0ad1ff8ca481a92504f38db23a0253ed30d38907611b87aaee38bc" gracePeriod=30
Oct 31 00:49:02.327144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:02.327026    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" podUID="e9f27690-bb8e-4159-b918-84baff256f09" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:49:03.127772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:03.127146    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:49:03.127772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:03.127233    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:49:03.151772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:03.142822    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:49:03.151772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:03.142912    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:49:03.316894 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:03.316846    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:49:03.338771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:03.320774    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:49:04.166107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.166053    4924 patch_prober.go:29] interesting pod/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:49:04.166411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.166366    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:49:04.167205 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.167176    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" containerID="cri-o://918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce" gracePeriod=45
Oct 31 00:49:04.543802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.535081    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"]
Oct 31 00:49:04.543802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.543423    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-5c2a03-6rbfq"]
Oct 31 00:49:04.548434 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.548403    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:49:04.636498 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.636455    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache\") pod \"2e29f410-c8ad-44ca-98a2-df59935478ef\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") "
Oct 31 00:49:04.636658 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.636563    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home\") pod \"2e29f410-c8ad-44ca-98a2-df59935478ef\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") "
Oct 31 00:49:04.636697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.636660    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs\" (UniqueName: \"kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs\") pod \"2e29f410-c8ad-44ca-98a2-df59935478ef\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") "
Oct 31 00:49:04.636854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.636835    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir\") pod \"2e29f410-c8ad-44ca-98a2-df59935478ef\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") "
Oct 31 00:49:04.636916 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.636875    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm\") pod \"2e29f410-c8ad-44ca-98a2-df59935478ef\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") "
Oct 31 00:49:04.637042 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.636929    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl\" (UniqueName: \"kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl\") pod \"2e29f410-c8ad-44ca-98a2-df59935478ef\" (UID: \"2e29f410-c8ad-44ca-98a2-df59935478ef\") "
Oct 31 00:49:04.639476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.639441    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache" pod "2e29f410-c8ad-44ca-98a2-df59935478ef" (UID: "2e29f410-c8ad-44ca-98a2-df59935478ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:04.639636 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.639617    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home" pod "2e29f410-c8ad-44ca-98a2-df59935478ef" (UID: "2e29f410-c8ad-44ca-98a2-df59935478ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:04.639784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.639767    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir" pod "2e29f410-c8ad-44ca-98a2-df59935478ef" (UID: "2e29f410-c8ad-44ca-98a2-df59935478ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:04.656799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.642834    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm" pod "2e29f410-c8ad-44ca-98a2-df59935478ef" (UID: "2e29f410-c8ad-44ca-98a2-df59935478ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:04.656799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.649267    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs" pod "2e29f410-c8ad-44ca-98a2-df59935478ef" (UID: "2e29f410-c8ad-44ca-98a2-df59935478ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:49:04.656799 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.649313    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl" pod "2e29f410-c8ad-44ca-98a2-df59935478ef" (UID: "2e29f410-c8ad-44ca-98a2-df59935478ef"). InnerVolumeSpecName "kube-api-access-dswhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:04.738722 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.738682    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:04.738722 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.738716    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:04.738722 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.738726    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dswhl\" (UniqueName: \"kubernetes.io/projected/2e29f410-c8ad-44ca-98a2-df59935478ef-kube-api-access-dswhl\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:04.739063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.738738    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:04.739063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.738785    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e29f410-c8ad-44ca-98a2-df59935478ef-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:04.739063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.738796    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e29f410-c8ad-44ca-98a2-df59935478ef-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:04.952361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.952322    4924 generic.go:356] "Generic (PLEG): container finished" podID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerID="918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce" exitCode=0
Oct 31 00:49:04.952544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.952380    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"
Oct 31 00:49:04.952544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.952407    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" event={"ID":"2e29f410-c8ad-44ca-98a2-df59935478ef","Type":"ContainerDied","Data":"918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce"}
Oct 31 00:49:04.952544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.952453    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" event={"ID":"2e29f410-c8ad-44ca-98a2-df59935478ef","Type":"ContainerDied","Data":"a1f291715194ffbc9d39d4f9674fa60606fdd115ad1d70dcb0526772f9d5f77b"}
Oct 31 00:49:04.952544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.952473    4924 scope.go:122] "RemoveContainer" containerID="918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce"
Oct 31 00:49:04.964613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.964586    4924 scope.go:122] "RemoveContainer" containerID="918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce"
Oct 31 00:49:04.964966 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:04.964937    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce\": container with ID starting with 918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce not found: ID does not exist" containerID="918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce"
Oct 31 00:49:04.965079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:04.965002    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce"} err="failed to get container status \"918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce\": rpc error: code = NotFound desc = could not find container \"918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce\": container with ID starting with 918840514c1addee02b7b8dde486778d25e0e9f51937e2a54a38e8ced1fd37ce not found: ID does not exist"
Oct 31 00:49:05.000622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.000586    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"]
Oct 31 00:49:05.005721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.005689    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2"]
Oct 31 00:49:05.304525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.304497    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:49:05.355272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.355227    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q\" (UniqueName: \"kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q\") pod \"e9f27690-bb8e-4159-b918-84baff256f09\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") "
Oct 31 00:49:05.355477 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.355389    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume\") pod \"e9f27690-bb8e-4159-b918-84baff256f09\" (UID: \"e9f27690-bb8e-4159-b918-84baff256f09\") "
Oct 31 00:49:05.356306 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.356274    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume" pod "e9f27690-bb8e-4159-b918-84baff256f09" (UID: "e9f27690-bb8e-4159-b918-84baff256f09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:05.360783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.360733    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q" pod "e9f27690-bb8e-4159-b918-84baff256f09" (UID: "e9f27690-bb8e-4159-b918-84baff256f09"). InnerVolumeSpecName "kube-api-access-8xf5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:05.457910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.457868    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xf5q\" (UniqueName: \"kubernetes.io/projected/e9f27690-bb8e-4159-b918-84baff256f09-kube-api-access-8xf5q\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:05.457910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.457909    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f27690-bb8e-4159-b918-84baff256f09-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:05.512631 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.512575    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-b433e6/distinct-2-w2-b433e6-kserve-7fdcfddb8c-zztt2" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.114:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:05.781945 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.775782    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e783ee8-1151-4009-b889-de32cb744cbc" path="/var/lib/kubelet/pods/0e783ee8-1151-4009-b889-de32cb744cbc/volumes"
Oct 31 00:49:05.781945 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.776462    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" path="/var/lib/kubelet/pods/2e29f410-c8ad-44ca-98a2-df59935478ef/volumes"
Oct 31 00:49:05.920541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.920314    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.970636    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.970767    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.970837    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.970891    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.970927    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.970970    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.971027    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.971102    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.971131    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket\") pod \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\" (UID: \"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb\") "
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.974455    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:05.975760 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.974672    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:05.980825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.978712    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:05.980825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.978909    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:05.981990 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.981960    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:05.982294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.982273    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:05.982857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.982835    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:05.987006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.986974    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "kube-api-access-2jl5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:05.993728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.993684    4924 generic.go:356] "Generic (PLEG): container finished" podID="e9f27690-bb8e-4159-b918-84baff256f09" containerID="486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7" exitCode=0
Oct 31 00:49:05.994023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.993881    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" event={"ID":"e9f27690-bb8e-4159-b918-84baff256f09","Type":"ContainerDied","Data":"486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7"}
Oct 31 00:49:05.994023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.993939    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf" event={"ID":"e9f27690-bb8e-4159-b918-84baff256f09","Type":"ContainerDied","Data":"0ee13b04a36c3e93aaec6c0623e372cffabcff606f03a78f2bed121fe3cdeb9e"}
Oct 31 00:49:05.994023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.993967    4924 scope.go:122] "RemoveContainer" containerID="486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7"
Oct 31 00:49:05.995033 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:05.994178    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"
Oct 31 00:49:06.002981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.002941    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy" pod "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" (UID: "b80533bd-0a09-4909-9ad8-d6ab1b0da4fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.015960 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.015922    4924 generic.go:356] "Generic (PLEG): container finished" podID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerID="4f3bd7c408cc401d8478de0c7fbfc58780e1e3b87b2a7243f08d5e82bd54f2f5" exitCode=0
Oct 31 00:49:06.016109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.016074    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" event={"ID":"6f4b1e70-835c-4f70-a519-46ff69973f9f","Type":"ContainerDied","Data":"4f3bd7c408cc401d8478de0c7fbfc58780e1e3b87b2a7243f08d5e82bd54f2f5"}
Oct 31 00:49:06.019957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.019475    4924 generic.go:356] "Generic (PLEG): container finished" podID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerID="75ee4f1be45a1682a8ea28bd5106b5756b48dd12eb457bf4eb3bb2d1f61fb70a" exitCode=0
Oct 31 00:49:06.019957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.019593    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" event={"ID":"d25a8ae5-a217-4859-ac56-9ec49996c29a","Type":"ContainerDied","Data":"75ee4f1be45a1682a8ea28bd5106b5756b48dd12eb457bf4eb3bb2d1f61fb70a"}
Oct 31 00:49:06.036415 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.036368    4924 scope.go:122] "RemoveContainer" containerID="486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7"
Oct 31 00:49:06.041602 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.041562    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"]
Oct 31 00:49:06.044881 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:06.044843    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7\": container with ID starting with 486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7 not found: ID does not exist" containerID="486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7"
Oct 31 00:49:06.045024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.044891    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7"} err="failed to get container status \"486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7\": rpc error: code = NotFound desc = could not find container \"486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7\": container with ID starting with 486f91490ff30f8d2e05af9b6e5074c4c171791161544da2899c0c69ceeb2bd7 not found: ID does not exist"
Oct 31 00:49:06.045024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.044921    4924 scope.go:122] "RemoveContainer" containerID="0eb53ad932d7c315d9c79ded36071c1131fb1a3ba0c4f4b4be4fc28f4d81e53c"
Oct 31 00:49:06.045299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.045265    4924 generic.go:356] "Generic (PLEG): container finished" podID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerID="7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea" exitCode=0
Oct 31 00:49:06.045424 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.045401    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" event={"ID":"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb","Type":"ContainerDied","Data":"7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea"}
Oct 31 00:49:06.045524 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.045508    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d" event={"ID":"b80533bd-0a09-4909-9ad8-d6ab1b0da4fb","Type":"ContainerDied","Data":"63e5e34cb65e020ffada570267b4191dece367761e09abea7f507e4901162341"}
Oct 31 00:49:06.045914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.045891    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"
Oct 31 00:49:06.049505 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.049472    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f99ac0-9d8c55b7f-lblkf"]
Oct 31 00:49:06.055642 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.055597    4924 generic.go:356] "Generic (PLEG): container finished" podID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerID="b0c693e04f0ad1ff8ca481a92504f38db23a0253ed30d38907611b87aaee38bc" exitCode=0
Oct 31 00:49:06.056038 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.056007    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" event={"ID":"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b","Type":"ContainerDied","Data":"b0c693e04f0ad1ff8ca481a92504f38db23a0253ed30d38907611b87aaee38bc"}
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073788    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073822    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-kube-api-access-2jl5w\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073841    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073862    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073878    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073893    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073907    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073924    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.074600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.073937    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.092072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.092042    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:49:06.094881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.094852    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:49:06.104289 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.104253    4924 scope.go:122] "RemoveContainer" containerID="7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea"
Oct 31 00:49:06.137194 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.137162    4924 scope.go:122] "RemoveContainer" containerID="7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea"
Oct 31 00:49:06.137919 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:06.137865    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea\": container with ID starting with 7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea not found: ID does not exist" containerID="7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea"
Oct 31 00:49:06.138072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.138042    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea"} err="failed to get container status \"7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea\": rpc error: code = NotFound desc = could not find container \"7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea\": container with ID starting with 7ad1f7653d153549aa4e9fc7ccf23d097e31dfac5017562a63c475c85dca2aea not found: ID does not exist"
Oct 31 00:49:06.159140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.159052    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"]
Oct 31 00:49:06.168063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.168023    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q7f2d"]
Oct 31 00:49:06.175541 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.175501    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.175707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.175558    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.175707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.175652    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.175707 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.175686    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.176012 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.175718    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.176012 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.175788    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.176201 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.176106    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.178666 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.177385    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.181288 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.181048    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:06.181580 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.181546    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.183623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.183604    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.183797 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.183780    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.183935 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.183921    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.184053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184039    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.184166 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184152    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.184271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184257    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.184374 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184359    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.184495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184481    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.184606 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184593    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.184818 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184800    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket\") pod \"d25a8ae5-a217-4859-ac56-9ec49996c29a\" (UID: \"d25a8ae5-a217-4859-ac56-9ec49996c29a\") "
Oct 31 00:49:06.184968 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.184952    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo\") pod \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\" (UID: \"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b\") "
Oct 31 00:49:06.185595 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.185572    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.185728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.185715    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.185881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.185867    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.192021    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.196459    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "kube-api-access-tvsmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.196687    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.196914    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.197096    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.197951    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.199955    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.199974    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.200054    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "kube-api-access-bx9sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.200096    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.200255 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.200233    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:06.201310 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.201165    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:06.203201 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.203154    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.213885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.213712    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy" pod "d25a8ae5-a217-4859-ac56-9ec49996c29a" (UID: "d25a8ae5-a217-4859-ac56-9ec49996c29a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.214056 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.213740    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token" pod "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" (UID: "7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291072    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291167    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291186    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291237    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291250    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291263    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvsmv\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-kube-api-access-tvsmv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291319    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291336    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291350    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25a8ae5-a217-4859-ac56-9ec49996c29a-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291366    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291382    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx9sp\" (UniqueName: \"kubernetes.io/projected/d25a8ae5-a217-4859-ac56-9ec49996c29a-kube-api-access-bx9sp\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291398    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291414    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291427    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25a8ae5-a217-4859-ac56-9ec49996c29a-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.291664 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.291440    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.327307 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.327274    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:49:06.392709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.392669    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.392709 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.392714    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.392829    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.392955    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.393005    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393146 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.393045    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393427 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.393397    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393502 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.393459    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.393581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.393544    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token\") pod \"6f4b1e70-835c-4f70-a519-46ff69973f9f\" (UID: \"6f4b1e70-835c-4f70-a519-46ff69973f9f\") "
Oct 31 00:49:06.394487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.393847    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.394487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.394169    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.394487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.394366    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.394487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.394443    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.394487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.394461    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.394725 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.394494    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:06.395378 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.395352    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:06.396931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.396889    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "kube-api-access-lncs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:06.397043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.396997    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:06.397216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.397191    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo" pod "6f4b1e70-835c-4f70-a519-46ff69973f9f" (UID: "6f4b1e70-835c-4f70-a519-46ff69973f9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:06.496009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.495907    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.495949    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.495963    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.495975    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4b1e70-835c-4f70-a519-46ff69973f9f-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.495987    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.496000    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.496016    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lncs9\" (UniqueName: \"kubernetes.io/projected/6f4b1e70-835c-4f70-a519-46ff69973f9f-kube-api-access-lncs9\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:06.496329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:06.496031    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4b1e70-835c-4f70-a519-46ff69973f9f-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:07.075529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.075501    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"
Oct 31 00:49:07.075529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.075514    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c" event={"ID":"6f4b1e70-835c-4f70-a519-46ff69973f9f","Type":"ContainerDied","Data":"db53cf888fdd557952033378e6efc0cb5cae1860acb31d3959a103edac81e375"}
Oct 31 00:49:07.076182 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.075578    4924 scope.go:122] "RemoveContainer" containerID="4f3bd7c408cc401d8478de0c7fbfc58780e1e3b87b2a7243f08d5e82bd54f2f5"
Oct 31 00:49:07.077938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.077913    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"
Oct 31 00:49:07.078100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.078046    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh" event={"ID":"d25a8ae5-a217-4859-ac56-9ec49996c29a","Type":"ContainerDied","Data":"dea90d14c6e8f17c6ed75df95e845dfb0d9906374016b2a400884e79006d9d48"}
Oct 31 00:49:07.089795 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.089778    4924 scope.go:122] "RemoveContainer" containerID="75ee4f1be45a1682a8ea28bd5106b5756b48dd12eb457bf4eb3bb2d1f61fb70a"
Oct 31 00:49:07.090346 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.090322    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9" event={"ID":"7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b","Type":"ContainerDied","Data":"3944ef19f4edb1b94b3803df484f417b844e28f616c905d9df457a79eeefe698"}
Oct 31 00:49:07.090457 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.090379    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"
Oct 31 00:49:07.104319 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.104294    4924 scope.go:122] "RemoveContainer" containerID="b0c693e04f0ad1ff8ca481a92504f38db23a0253ed30d38907611b87aaee38bc"
Oct 31 00:49:07.121241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.121214    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"]
Oct 31 00:49:07.128332 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.128308    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-q622c"]
Oct 31 00:49:07.137405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.137380    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"]
Oct 31 00:49:07.142508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.142484    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-x2mjh"]
Oct 31 00:49:07.153667 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.153629    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"]
Oct 31 00:49:07.159416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.159391    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f99ac0-openshift-default-84bc7fcc97-qgbv9"]
Oct 31 00:49:07.766102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.766062    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" path="/var/lib/kubelet/pods/6f4b1e70-835c-4f70-a519-46ff69973f9f/volumes"
Oct 31 00:49:07.766728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.766710    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" path="/var/lib/kubelet/pods/7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b/volumes"
Oct 31 00:49:07.767329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.767312    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" path="/var/lib/kubelet/pods/b80533bd-0a09-4909-9ad8-d6ab1b0da4fb/volumes"
Oct 31 00:49:07.768141 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.768126    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" path="/var/lib/kubelet/pods/d25a8ae5-a217-4859-ac56-9ec49996c29a/volumes"
Oct 31 00:49:07.768494 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:07.768481    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9f27690-bb8e-4159-b918-84baff256f09" path="/var/lib/kubelet/pods/e9f27690-bb8e-4159-b918-84baff256f09/volumes"
Oct 31 00:49:10.861312 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:10.861274    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"]
Oct 31 00:49:10.870147 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:10.870110    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b433e6-tfhpw"]
Oct 31 00:49:11.765045 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.765006    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da5e3e23-760a-44d5-a4b3-5806dadefcab" path="/var/lib/kubelet/pods/da5e3e23-760a-44d5-a4b3-5806dadefcab/volumes"
Oct 31 00:49:11.879063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.878980    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"]
Oct 31 00:49:11.879607 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.879273    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy" containerID="cri-o://50dc87b7e3d795eda116139637cb51b50184fab1597b6d9eb45b98e67ba41f3f" gracePeriod=30
Oct 31 00:49:11.892336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.891369    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"]
Oct 31 00:49:11.892336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.891678    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy" containerID="cri-o://c83f09c67a49ae6a507156d614973b43d2801546d8729031df23056fe0c4ee81" gracePeriod=30
Oct 31 00:49:11.899789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.899738    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"]
Oct 31 00:49:11.900016 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.899993    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy" containerID="cri-o://7e1ede373e466bfebc9a54cf6b3b0481f898601b3ceb08d45e7ddad37a3e533e" gracePeriod=30
Oct 31 00:49:11.907235 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.907202    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"]
Oct 31 00:49:11.907465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:11.907440    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" containerID="cri-o://dbe36733f7b7424cf1ea8e96ff575daa817529cf3564f8de0e1d221d6bd00c9f" gracePeriod=30
Oct 31 00:49:13.261039 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:13.261000    4924 patch_prober.go:29] interesting pod/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:49:13.261496 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:13.261079    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:49:13.288224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:13.288185    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:49:13.288426 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:13.288399    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:49:17.197550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.196417    4924 generic.go:356] "Generic (PLEG): container finished" podID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerID="c83f09c67a49ae6a507156d614973b43d2801546d8729031df23056fe0c4ee81" exitCode=0
Oct 31 00:49:17.197550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.196557    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" event={"ID":"8677e702-1ca1-469c-bdd7-76e4f1512c9c","Type":"ContainerDied","Data":"c83f09c67a49ae6a507156d614973b43d2801546d8729031df23056fe0c4ee81"}
Oct 31 00:49:17.199852 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.198850    4924 generic.go:356] "Generic (PLEG): container finished" podID="914a5656-be09-46ed-8413-633981153991" containerID="50dc87b7e3d795eda116139637cb51b50184fab1597b6d9eb45b98e67ba41f3f" exitCode=0
Oct 31 00:49:17.199852 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.198930    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" event={"ID":"914a5656-be09-46ed-8413-633981153991","Type":"ContainerDied","Data":"50dc87b7e3d795eda116139637cb51b50184fab1597b6d9eb45b98e67ba41f3f"}
Oct 31 00:49:17.199852 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.199014    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk" event={"ID":"914a5656-be09-46ed-8413-633981153991","Type":"ContainerDied","Data":"aad8da2f607c87fee7e4339233f6455e2028c8ed909ed29b7a7887251aff9ccb"}
Oct 31 00:49:17.199852 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.199039    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aad8da2f607c87fee7e4339233f6455e2028c8ed909ed29b7a7887251aff9ccb"
Oct 31 00:49:17.202719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.201576    4924 generic.go:356] "Generic (PLEG): container finished" podID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerID="dbe36733f7b7424cf1ea8e96ff575daa817529cf3564f8de0e1d221d6bd00c9f" exitCode=0
Oct 31 00:49:17.202719 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.201789    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" event={"ID":"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d","Type":"ContainerDied","Data":"dbe36733f7b7424cf1ea8e96ff575daa817529cf3564f8de0e1d221d6bd00c9f"}
Oct 31 00:49:17.207155 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.204880    4924 generic.go:356] "Generic (PLEG): container finished" podID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerID="7e1ede373e466bfebc9a54cf6b3b0481f898601b3ceb08d45e7ddad37a3e533e" exitCode=0
Oct 31 00:49:17.207155 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.204982    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" event={"ID":"5ae76819-9bd6-4f27-9fa9-b63ff98440fb","Type":"ContainerDied","Data":"7e1ede373e466bfebc9a54cf6b3b0481f898601b3ceb08d45e7ddad37a3e533e"}
Oct 31 00:49:17.207155 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.206048    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:49:17.237013 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.236976    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:49:17.306318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.306291    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:49:17.312237 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.312216    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:49:17.341436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341397    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.341622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341457    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.341622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341602    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.341726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341652    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.341726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341698    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.341865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341780    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.341865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341831    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.341979 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341885    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.341979 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.341957    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.342079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342038    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.342079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342069    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.342199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342119    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:17.342199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342164    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.342299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342201    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.342299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342232    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.342299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342265    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy\") pod \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\" (UID: \"8677e702-1ca1-469c-bdd7-76e4f1512c9c\") "
Oct 31 00:49:17.342299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342293    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.342508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342351    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.342508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342378    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo\") pod \"914a5656-be09-46ed-8413-633981153991\" (UID: \"914a5656-be09-46ed-8413-633981153991\") "
Oct 31 00:49:17.342508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342389    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.342681 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342554    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.342940 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342916    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914a5656-be09-46ed-8413-633981153991-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.343044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342943    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.343044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.342957    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.343160 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.343126    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.343867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.343317    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.343867 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.343568    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.344139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.344022    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:17.344252 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.344222    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "kube-api-access-dpqw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.344331 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.344306    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.344539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.344509    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.344539 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.344531    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.344808 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.344785    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.345237 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.345210    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "kube-api-access-vqhkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.346286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.346262    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:17.346837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.346800    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:17.349160 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.349138    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.349963 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.349930    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy" pod "8677e702-1ca1-469c-bdd7-76e4f1512c9c" (UID: "8677e702-1ca1-469c-bdd7-76e4f1512c9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.351788 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.350847    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy" pod "914a5656-be09-46ed-8413-633981153991" (UID: "914a5656-be09-46ed-8413-633981153991"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.443673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443594    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.443673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443633    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.443673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443655    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443694    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443759    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443789    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443817    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443859    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443884    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443950    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443970    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.443986 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.443989    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444018    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444065    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444101    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444180    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444211    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444224    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444242    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444265    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs\") pod \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\" (UID: \"5ae76819-9bd6-4f27-9fa9-b63ff98440fb\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444304    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444335    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy\") pod \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\" (UID: \"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d\") "
Oct 31 00:49:17.444510 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444398    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.445102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444523    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.445102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444580    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.445102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.444988    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445013    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445027    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445167    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445190    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqhkt\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-kube-api-access-vqhkt\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445205    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445220    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445236    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445250    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445266    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445281    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445296    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914a5656-be09-46ed-8413-633981153991-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445310    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445327    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445341    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914a5656-be09-46ed-8413-633981153991-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445355    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914a5656-be09-46ed-8413-633981153991-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445372    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpqw5\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-kube-api-access-dpqw5\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445387    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.445395 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445402    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.446258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445418    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.446258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445436    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8677e702-1ca1-469c-bdd7-76e4f1512c9c-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.446258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445450    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.446258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445710    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.446258 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.445935    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.446943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.446678    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "kube-api-access-2v9kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.447071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.446956    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.447071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.446979    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:17.447071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.447010    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.447071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.447054    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.447234 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.447157    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:17.447273 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.447249    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.447489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.447469    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd" pod "5ae76819-9bd6-4f27-9fa9-b63ff98440fb" (UID: "5ae76819-9bd6-4f27-9fa9-b63ff98440fb"). InnerVolumeSpecName "kube-api-access-7sbnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:17.447985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.447969    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo" pod "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" (UID: "4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:17.547094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547055    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sbnd\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-kube-api-access-7sbnd\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547088    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547097    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547106    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547114    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547122    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547130    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547138    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ae76819-9bd6-4f27-9fa9-b63ff98440fb-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547146    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547154    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:17.547333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:17.547164    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v9kf\" (UniqueName: \"kubernetes.io/projected/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d-kube-api-access-2v9kf\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:18.013376 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.013338    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"]
Oct 31 00:49:18.013627 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.013599    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy" containerID="cri-o://d1af9c4211e5eaf2470891324ab134516bc614785cfb3b2a9e6d79e9103c20e5" gracePeriod=30
Oct 31 00:49:18.019633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.019598    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"]
Oct 31 00:49:18.019856 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.019821    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" containerID="cri-o://03e1d1baa5edf4db57e9f49937e8639a4ceb744134e88e7424880516e0fc41c1" gracePeriod=30
Oct 31 00:49:18.048485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.048400    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"]
Oct 31 00:49:18.048621 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.048597    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" containerID="cri-o://342028bb9114c6b178acae2c95934f13cd0ad5b1036e5f56dd2c98ea23a6aabf" gracePeriod=30
Oct 31 00:49:18.215732 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.215701    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"
Oct 31 00:49:18.216240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.215727    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd" event={"ID":"8677e702-1ca1-469c-bdd7-76e4f1512c9c","Type":"ContainerDied","Data":"b1a1dc900cda1226e3553ffa8bfd1606cf9f4f62c98007f8a5e457493ca225a1"}
Oct 31 00:49:18.216240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.215819    4924 scope.go:122] "RemoveContainer" containerID="c83f09c67a49ae6a507156d614973b43d2801546d8729031df23056fe0c4ee81"
Oct 31 00:49:18.218399 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.218374    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq" event={"ID":"4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d","Type":"ContainerDied","Data":"5598ea3ed2bc72ad40e32a58d24d128ce442877d9ef1b974c5e0d33edf5614fb"}
Oct 31 00:49:18.218548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.218475    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"
Oct 31 00:49:18.221238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.221217    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"
Oct 31 00:49:18.221379 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.221302    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg" event={"ID":"5ae76819-9bd6-4f27-9fa9-b63ff98440fb","Type":"ContainerDied","Data":"f7ab4f11be0194cb2b84de7adfa5b0f4a83f12849ce51835fe708cb3471863a4"}
Oct 31 00:49:18.221446 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.221438    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"
Oct 31 00:49:18.234549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.234518    4924 scope.go:122] "RemoveContainer" containerID="dbe36733f7b7424cf1ea8e96ff575daa817529cf3564f8de0e1d221d6bd00c9f"
Oct 31 00:49:18.246102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.246075    4924 scope.go:122] "RemoveContainer" containerID="7e1ede373e466bfebc9a54cf6b3b0481f898601b3ceb08d45e7ddad37a3e533e"
Oct 31 00:49:18.249653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.249627    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"]
Oct 31 00:49:18.256716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.256691    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-g69bd"]
Oct 31 00:49:18.273092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.273051    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"]
Oct 31 00:49:18.283131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.283084    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-997qq"]
Oct 31 00:49:18.294044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.293985    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"]
Oct 31 00:49:18.299247 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.299167    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-whkpk"]
Oct 31 00:49:18.309756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.309707    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"]
Oct 31 00:49:18.313917 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:18.313895    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-5c2a03-openshift-default-686489c6b5-dpxrg"]
Oct 31 00:49:19.767443 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:19.767405    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" path="/var/lib/kubelet/pods/4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d/volumes"
Oct 31 00:49:19.768109 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:19.768086    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" path="/var/lib/kubelet/pods/5ae76819-9bd6-4f27-9fa9-b63ff98440fb/volumes"
Oct 31 00:49:19.768553 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:19.768538    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" path="/var/lib/kubelet/pods/8677e702-1ca1-469c-bdd7-76e4f1512c9c/volumes"
Oct 31 00:49:19.769315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:19.769300    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="914a5656-be09-46ed-8413-633981153991" path="/var/lib/kubelet/pods/914a5656-be09-46ed-8413-633981153991/volumes"
Oct 31 00:49:19.991775 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:19.991732    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:49:20.015873 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.015834    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:49:20.176622 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.176584    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume\") pod \"cb5f27fd-5a43-49b3-8644-8096e11165ed\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") "
Oct 31 00:49:20.176849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.176719    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume\") pod \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") "
Oct 31 00:49:20.176849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.176817    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm\" (UniqueName: \"kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm\") pod \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\" (UID: \"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7\") "
Oct 31 00:49:20.176996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.176917    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl\" (UniqueName: \"kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl\") pod \"cb5f27fd-5a43-49b3-8644-8096e11165ed\" (UID: \"cb5f27fd-5a43-49b3-8644-8096e11165ed\") "
Oct 31 00:49:20.177104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.177075    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume" pod "cb5f27fd-5a43-49b3-8644-8096e11165ed" (UID: "cb5f27fd-5a43-49b3-8644-8096e11165ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:20.177151 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.177087    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume" pod "23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" (UID: "23cea57f-ad09-4f8d-9638-b2b1b7cce1b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:20.177285 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.177267    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5f27fd-5a43-49b3-8644-8096e11165ed-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:20.177343 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.177292    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:20.179199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.179174    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl" pod "cb5f27fd-5a43-49b3-8644-8096e11165ed" (UID: "cb5f27fd-5a43-49b3-8644-8096e11165ed"). InnerVolumeSpecName "kube-api-access-4hpzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:20.179199 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.179183    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm" pod "23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" (UID: "23cea57f-ad09-4f8d-9638-b2b1b7cce1b7"). InnerVolumeSpecName "kube-api-access-8l8nm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:20.255798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.255677    4924 generic.go:356] "Generic (PLEG): container finished" podID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerID="4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb" exitCode=0
Oct 31 00:49:20.255798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.255728    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"
Oct 31 00:49:20.255798 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.255782    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" event={"ID":"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7","Type":"ContainerDied","Data":"4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb"}
Oct 31 00:49:20.256078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.255848    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8" event={"ID":"23cea57f-ad09-4f8d-9638-b2b1b7cce1b7","Type":"ContainerDied","Data":"cf0d520354aa20078f87d042eb45488f2bf68b30ee8b1d68d3b8b2ccb0e33063"}
Oct 31 00:49:20.256078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.255884    4924 scope.go:122] "RemoveContainer" containerID="4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb"
Oct 31 00:49:20.257437 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.257415    4924 generic.go:356] "Generic (PLEG): container finished" podID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerID="59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121" exitCode=0
Oct 31 00:49:20.257568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.257466    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"
Oct 31 00:49:20.257568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.257504    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" event={"ID":"cb5f27fd-5a43-49b3-8644-8096e11165ed","Type":"ContainerDied","Data":"59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121"}
Oct 31 00:49:20.257568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.257549    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf" event={"ID":"cb5f27fd-5a43-49b3-8644-8096e11165ed","Type":"ContainerDied","Data":"5d06e8b5543f7480bba8c76888dcc66418b2df95f8d42b05fe6332e7fa81b6f8"}
Oct 31 00:49:20.268804 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.268781    4924 scope.go:122] "RemoveContainer" containerID="4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb"
Oct 31 00:49:20.269144 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:20.269121    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb\": container with ID starting with 4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb not found: ID does not exist" containerID="4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb"
Oct 31 00:49:20.269206 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.269153    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb"} err="failed to get container status \"4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb\": rpc error: code = NotFound desc = could not find container \"4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb\": container with ID starting with 4d8db1b5e50ff5d4419179ba786d2d0c4015fa56d40be75d1e10519ddfff96eb not found: ID does not exist"
Oct 31 00:49:20.269206 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.269179    4924 scope.go:122] "RemoveContainer" containerID="59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121"
Oct 31 00:49:20.278614 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.278592    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hpzl\" (UniqueName: \"kubernetes.io/projected/cb5f27fd-5a43-49b3-8644-8096e11165ed-kube-api-access-4hpzl\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:20.278697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.278617    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l8nm\" (UniqueName: \"kubernetes.io/projected/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7-kube-api-access-8l8nm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:20.281057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.281037    4924 scope.go:122] "RemoveContainer" containerID="59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121"
Oct 31 00:49:20.281348 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:20.281325    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121\": container with ID starting with 59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121 not found: ID does not exist" containerID="59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121"
Oct 31 00:49:20.281427 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.281355    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121"} err="failed to get container status \"59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121\": rpc error: code = NotFound desc = could not find container \"59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121\": container with ID starting with 59c5b63c186d9dcbd165441b53e1484f8195e2628ffb5d54483e77c4ad4ee121 not found: ID does not exist"
Oct 31 00:49:20.287407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.287382    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"]
Oct 31 00:49:20.291715 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.291691    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b433e6-7f98f98dbc-vzhlf"]
Oct 31 00:49:20.301874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.301847    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"]
Oct 31 00:49:20.305979 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:20.305957    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b433e6-5c4fdcff7c-c5jp8"]
Oct 31 00:49:21.218238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:21.218195    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:49:21.218764 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:21.218282    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:49:21.765385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:21.765339    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" path="/var/lib/kubelet/pods/23cea57f-ad09-4f8d-9638-b2b1b7cce1b7/volumes"
Oct 31 00:49:21.765934 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:21.765915    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" path="/var/lib/kubelet/pods/cb5f27fd-5a43-49b3-8644-8096e11165ed/volumes"
Oct 31 00:49:22.781497 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:22.781458    4924 patch_prober.go:29] interesting pod/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:49:22.781985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:22.781527    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:49:23.323947 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.323312    4924 generic.go:356] "Generic (PLEG): container finished" podID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerID="d1af9c4211e5eaf2470891324ab134516bc614785cfb3b2a9e6d79e9103c20e5" exitCode=0
Oct 31 00:49:23.323947 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.323425    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" event={"ID":"847e6d33-4e12-49e7-96c3-7f2bc41300fc","Type":"ContainerDied","Data":"d1af9c4211e5eaf2470891324ab134516bc614785cfb3b2a9e6d79e9103c20e5"}
Oct 31 00:49:23.326256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.326052    4924 generic.go:356] "Generic (PLEG): container finished" podID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerID="03e1d1baa5edf4db57e9f49937e8639a4ceb744134e88e7424880516e0fc41c1" exitCode=0
Oct 31 00:49:23.326256 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.326177    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" event={"ID":"e6643d43-f92c-4820-b206-cd6e78a7c3b5","Type":"ContainerDied","Data":"03e1d1baa5edf4db57e9f49937e8639a4ceb744134e88e7424880516e0fc41c1"}
Oct 31 00:49:23.329284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.329053    4924 generic.go:356] "Generic (PLEG): container finished" podID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerID="342028bb9114c6b178acae2c95934f13cd0ad5b1036e5f56dd2c98ea23a6aabf" exitCode=0
Oct 31 00:49:23.329284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.329127    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" event={"ID":"a2b570d4-6c8e-4454-a84a-37c4c4118471","Type":"ContainerDied","Data":"342028bb9114c6b178acae2c95934f13cd0ad5b1036e5f56dd2c98ea23a6aabf"}
Oct 31 00:49:23.329284 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.329177    4924 scope.go:122] "RemoveContainer" containerID="ce4b1a13c849fcc6eb2c8fe71cf1c7c698c9cfb3e806d5adb81960689a74d0f2"
Oct 31 00:49:23.442907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.430816    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524455    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524527    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524591    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524621    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524654    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524684    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524791    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524856    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.524915    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket\") pod \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\" (UID: \"847e6d33-4e12-49e7-96c3-7f2bc41300fc\") "
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.525560    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.525799    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.526543    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.529365    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.541769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.529910    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:23.620590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.620163    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.625596    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.625646    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.625723    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.625903    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.625945    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626076    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626108    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626152    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626191    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token\") pod \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\" (UID: \"e6643d43-f92c-4820-b206-cd6e78a7c3b5\") "
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626586    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626608    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626625    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626639    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.627065 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.626655    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.628550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.628152    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:23.631682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.631459    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:23.632243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.631998    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:23.632582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.632401    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.632769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.632714    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.633350 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.633056    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.637890 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.637293    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:23.638247 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.638201    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.638865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.638826    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.664716 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.664678    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489" pod "e6643d43-f92c-4820-b206-cd6e78a7c3b5" (UID: "e6643d43-f92c-4820-b206-cd6e78a7c3b5"). InnerVolumeSpecName "kube-api-access-76489". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:23.668298 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.668071    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "kube-api-access-2ccnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:23.668298 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.668225    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:23.670599 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.670338    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy" pod "847e6d33-4e12-49e7-96c3-7f2bc41300fc" (UID: "847e6d33-4e12-49e7-96c3-7f2bc41300fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.671871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.671473    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:49:23.733938 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.733873    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.733941    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.733981    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734132 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734113    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734152    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734190    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734294 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734239    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.734680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734584    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734609    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734625    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734641    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ccnd\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-kube-api-access-2ccnd\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734659    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76489\" (UniqueName: \"kubernetes.io/projected/e6643d43-f92c-4820-b206-cd6e78a7c3b5-kube-api-access-76489\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734673    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734727    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734741    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734771    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734785    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6643d43-f92c-4820-b206-cd6e78a7c3b5-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734799    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847e6d33-4e12-49e7-96c3-7f2bc41300fc-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734815    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.734981 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.734830    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6643d43-f92c-4820-b206-cd6e78a7c3b5-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.735370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.735070    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.735783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.735479    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.736420 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.736375    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.736843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.736822    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:49:23.737998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.737964    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:23.740012 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.739982    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:49:23.740909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.740880    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835410    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835477    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx\") pod \"a2b570d4-6c8e-4454-a84a-37c4c4118471\" (UID: \"a2b570d4-6c8e-4454-a84a-37c4c4118471\") "
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835770    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835940    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b570d4-6c8e-4454-a84a-37c4c4118471-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835960    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835976    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.835989    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.836004    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.836019    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.836033    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.836202 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.836047    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b570d4-6c8e-4454-a84a-37c4c4118471-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:23.840813 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.840728    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx" pod "a2b570d4-6c8e-4454-a84a-37c4c4118471" (UID: "a2b570d4-6c8e-4454-a84a-37c4c4118471"). InnerVolumeSpecName "kube-api-access-vzvxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:49:23.937336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:23.937223    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzvxx\" (UniqueName: \"kubernetes.io/projected/a2b570d4-6c8e-4454-a84a-37c4c4118471-kube-api-access-vzvxx\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:49:24.391772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.391062    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp" event={"ID":"847e6d33-4e12-49e7-96c3-7f2bc41300fc","Type":"ContainerDied","Data":"d0ae581692532dee2583a76626697ec729d6ea094484ac6e76cd25952360eff5"}
Oct 31 00:49:24.391772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.391146    4924 scope.go:122] "RemoveContainer" containerID="d1af9c4211e5eaf2470891324ab134516bc614785cfb3b2a9e6d79e9103c20e5"
Oct 31 00:49:24.391772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.391284    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"
Oct 31 00:49:24.476540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.476457    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw" event={"ID":"e6643d43-f92c-4820-b206-cd6e78a7c3b5","Type":"ContainerDied","Data":"57d10b7fa595ff3905ae1008bc836ade9e988b4f224e175d37e7b89515d85dea"}
Oct 31 00:49:24.476784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.476660    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"
Oct 31 00:49:24.480970 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.479569    4924 scope.go:122] "RemoveContainer" containerID="03e1d1baa5edf4db57e9f49937e8639a4ceb744134e88e7424880516e0fc41c1"
Oct 31 00:49:24.504899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.504807    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"]
Oct 31 00:49:24.517105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.517067    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6" event={"ID":"a2b570d4-6c8e-4454-a84a-37c4c4118471","Type":"ContainerDied","Data":"93c3e921973890ccfc599ed28844ef29a3f2a424e23892cf4038155f53bfcda4"}
Oct 31 00:49:24.517228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.517079    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"
Oct 31 00:49:24.548771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.547397    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-dkxgp"]
Oct 31 00:49:24.632003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.623797    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"]
Oct 31 00:49:24.638790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.635247    4924 scope.go:122] "RemoveContainer" containerID="342028bb9114c6b178acae2c95934f13cd0ad5b1036e5f56dd2c98ea23a6aabf"
Oct 31 00:49:24.679861 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.675810    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-7sstw"]
Oct 31 00:49:24.701156 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.701072    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"]
Oct 31 00:49:24.732785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:24.728872    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-b433e6-openshift-default-755cb85cc5-jhxv6"]
Oct 31 00:49:25.307899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:25.307859    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:25.308478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:25.308441    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:25.800770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:25.799872    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" path="/var/lib/kubelet/pods/847e6d33-4e12-49e7-96c3-7f2bc41300fc/volumes"
Oct 31 00:49:25.800770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:25.800548    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" path="/var/lib/kubelet/pods/a2b570d4-6c8e-4454-a84a-37c4c4118471/volumes"
Oct 31 00:49:25.823771 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:49:25.821558    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" path="/var/lib/kubelet/pods/e6643d43-f92c-4820-b206-cd6e78a7c3b5/volumes"
Oct 31 00:49:27.852387 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:27.852352    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:49:57.853321 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:49:57.853287    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:50:12.025700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:12.025659    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"]
Oct 31 00:50:12.031913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:12.031871    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6205a64c-n2g5w"]
Oct 31 00:50:13.764921 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:13.764885    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bce8f095-7fc6-4094-885d-874962f41717" path="/var/lib/kubelet/pods/bce8f095-7fc6-4094-885d-874962f41717/volumes"
Oct 31 00:50:20.025271 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:20.025234    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"]
Oct 31 00:50:20.029696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:20.029670    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3fab882f-lg2tl"]
Oct 31 00:50:21.764130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:21.764091    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="139c87ed-b8ff-484c-b471-5684a5023a5b" path="/var/lib/kubelet/pods/139c87ed-b8ff-484c-b471-5684a5023a5b/volumes"
Oct 31 00:50:23.024643 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:23.024604    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"]
Oct 31 00:50:23.029509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:23.029484    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-709031f8-tr98z"]
Oct 31 00:50:23.763573 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:23.763539    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d895f4e-1d56-41ff-b99a-3959a00bc5e9" path="/var/lib/kubelet/pods/9d895f4e-1d56-41ff-b99a-3959a00bc5e9/volumes"
Oct 31 00:50:26.291946 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:26.291911    4924 scope.go:122] "RemoveContainer" containerID="50dc87b7e3d795eda116139637cb51b50184fab1597b6d9eb45b98e67ba41f3f"
Oct 31 00:50:26.304095 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:26.304073    4924 scope.go:122] "RemoveContainer" containerID="bb29103e6a8f0307228b7b7bf4e459917216b7a40b7d089bdc59258538278290"
Oct 31 00:50:27.854046 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:50:27.854006    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:50:33.032189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:33.032098    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"]
Oct 31 00:50:33.041542 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:33.041467    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"]
Oct 31 00:50:33.050857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:33.050801    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec21c7ce-bxdkn"]
Oct 31 00:50:33.056133 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:33.056103    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cc4c7b-q62tn"]
Oct 31 00:50:33.763471 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:33.763434    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="baeee1db-8c68-443f-90c3-3be9702a6b8d" path="/var/lib/kubelet/pods/baeee1db-8c68-443f-90c3-3be9702a6b8d/volumes"
Oct 31 00:50:33.764000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:33.763978    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7" path="/var/lib/kubelet/pods/ebb2bc97-f1a9-4efa-b0cf-b85f4388f6f7/volumes"
Oct 31 00:50:34.028099 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:34.028063    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"]
Oct 31 00:50:34.032244 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:34.032214    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f81abd9-22mn5"]
Oct 31 00:50:35.763034 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:35.763001    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f0bb425-505d-4ba2-80e8-b256b559b46e" path="/var/lib/kubelet/pods/9f0bb425-505d-4ba2-80e8-b256b559b46e/volumes"
Oct 31 00:50:37.023991 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:37.023937    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"]
Oct 31 00:50:37.027669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:37.027645    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e97ee4ae-hb9md"]
Oct 31 00:50:37.763002 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:37.762968    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32833fc2-4d9f-4731-bed2-8112b9bc67ff" path="/var/lib/kubelet/pods/32833fc2-4d9f-4731-bed2-8112b9bc67ff/volumes"
Oct 31 00:50:43.022996 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:43.022959    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"]
Oct 31 00:50:43.026679 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:43.026655    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47059f59-vpzxc"]
Oct 31 00:50:43.763666 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:43.763633    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68a3cf8a-e155-4b83-9b09-4983a0bda04f" path="/var/lib/kubelet/pods/68a3cf8a-e155-4b83-9b09-4983a0bda04f/volumes"
Oct 31 00:50:45.024508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:45.024467    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"]
Oct 31 00:50:45.028283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:45.028253    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78472ed3-hkgjt"]
Oct 31 00:50:45.762851 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:45.762816    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9555a8e3-c99a-439c-a878-c213d78e3068" path="/var/lib/kubelet/pods/9555a8e3-c99a-439c-a878-c213d78e3068/volumes"
Oct 31 00:50:47.028419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:47.028360    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"]
Oct 31 00:50:47.036503 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:47.033572    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a503710-5jjzt"]
Oct 31 00:50:47.763241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:47.763206    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69083ca8-f0bb-4c02-88b7-e944b79b39a6" path="/var/lib/kubelet/pods/69083ca8-f0bb-4c02-88b7-e944b79b39a6/volumes"
Oct 31 00:50:48.022735 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:48.022697    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"]
Oct 31 00:50:48.028268 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:48.028238    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd0199c3-fhlh9"]
Oct 31 00:50:49.764088 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:49.764056    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86bbcd08-30c2-42aa-a053-e55887c428a5" path="/var/lib/kubelet/pods/86bbcd08-30c2-42aa-a053-e55887c428a5/volumes"
Oct 31 00:50:52.753006 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:52.752959    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:50:52.754682 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:52.754646    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:57.023740 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:57.023698    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"]
Oct 31 00:50:57.027289 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:57.027263    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-aa9f6c91-svxmq"]
Oct 31 00:50:57.763342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:57.763305    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86404c9f-c131-436a-94c9-8da6c9a752be" path="/var/lib/kubelet/pods/86404c9f-c131-436a-94c9-8da6c9a752be/volumes"
Oct 31 00:50:57.853427 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:50:57.853388    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:50:58.024072 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:58.024020    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"]
Oct 31 00:50:58.029275 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:58.029247    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d517c3c3-ph785"]
Oct 31 00:50:59.763550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:50:59.763516    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1da422dc-6ecc-4c70-a01f-1ac212ee9944" path="/var/lib/kubelet/pods/1da422dc-6ecc-4c70-a01f-1ac212ee9944/volumes"
Oct 31 00:51:15.026974 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:15.026933    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"]
Oct 31 00:51:15.030785 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:15.030735    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c01141f6-s6jzz"]
Oct 31 00:51:15.765855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:15.765812    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf9eff68-370b-44ef-8d0f-6cc1715c79f8" path="/var/lib/kubelet/pods/cf9eff68-370b-44ef-8d0f-6cc1715c79f8/volumes"
Oct 31 00:51:22.043529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:22.043440    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"]
Oct 31 00:51:22.058892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:22.058829    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bea488a5-rppfq"]
Oct 31 00:51:22.728409 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:22.727864    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:51:22.728409 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:22.727952    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:51:23.763261 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:23.763229    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cf0b95f-fbed-4e37-b599-e164e93c17c9" path="/var/lib/kubelet/pods/6cf0b95f-fbed-4e37-b599-e164e93c17c9/volumes"
Oct 31 00:51:26.338082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:26.338049    4924 scope.go:122] "RemoveContainer" containerID="a8dbf56993e4b6aab83f8523969aea2fe1683feb0ddda14edc358813a9b82b4b"
Oct 31 00:51:26.348702 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:26.348676    4924 scope.go:122] "RemoveContainer" containerID="cabe72de564f5384692d3e427975e16062f7ed7aeb2fde0765e36bf45fcd7360"
Oct 31 00:51:26.358562 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:26.358536    4924 scope.go:122] "RemoveContainer" containerID="a945447e3d4a60996911c04663d848857a138cfcace2fde103aee8ad3271407e"
Oct 31 00:51:27.863665 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:51:27.863625    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:51:42.023918 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:42.023876    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"]
Oct 31 00:51:42.027616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:42.027579    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef46cd9e-2c4gm"]
Oct 31 00:51:43.766087 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:43.766046    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3206258-f745-4418-8be2-24a13fee1dfb" path="/var/lib/kubelet/pods/c3206258-f745-4418-8be2-24a13fee1dfb/volumes"
Oct 31 00:51:57.028898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:57.028816    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"]
Oct 31 00:51:57.034385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:57.034342    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d2c8acf-wvtkq"]
Oct 31 00:51:57.808485 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:51:57.808442    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa190b51-33ce-47e2-b931-04d63781068a" path="/var/lib/kubelet/pods/fa190b51-33ce-47e2-b931-04d63781068a/volumes"
Oct 31 00:51:58.001201 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:51:58.001162    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:52:20.049330 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:20.049226    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"]
Oct 31 00:52:20.056404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:20.056221    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc73a80c-dvgcn"]
Oct 31 00:52:21.764004 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:21.763966    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54bac04b-501a-4acb-bb7b-6ee4f2f483a6" path="/var/lib/kubelet/pods/54bac04b-501a-4acb-bb7b-6ee4f2f483a6/volumes"
Oct 31 00:52:26.399835 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.399801    4924 scope.go:122] "RemoveContainer" containerID="479ea9508faca66d776971888731df71e808365664e5b2fcd89bf8f443a20093"
Oct 31 00:52:26.409901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.409880    4924 scope.go:122] "RemoveContainer" containerID="c38b7272e7c30523c9ab90d182d57869854f72c7fdfc12f32ddd123110f25a9c"
Oct 31 00:52:26.418922 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.418895    4924 scope.go:122] "RemoveContainer" containerID="68fab8876983d02eaf5e6f2b91767d14a9dc936db709af7fec019add9021bd2c"
Oct 31 00:52:26.431188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.431162    4924 scope.go:122] "RemoveContainer" containerID="c616192ec1f71c81c2fddf238e61e2871fd57bf17e5611dc6f8590b1e5581ec6"
Oct 31 00:52:26.440370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.440350    4924 scope.go:122] "RemoveContainer" containerID="fae45bc0b24b4ae8a6074d41ff5aa8503d8f588014273ea1584c0f30bb320ea6"
Oct 31 00:52:26.449269 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.449249    4924 scope.go:122] "RemoveContainer" containerID="e76fe878d7b35519391b31cd9b4f70fa0fad93972ce9a0a2d8438e05e6e2fadc"
Oct 31 00:52:26.458115 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.458088    4924 scope.go:122] "RemoveContainer" containerID="0ba8e497071218ae2b3d09d2b6b0c053c66a8117e28568935a4567e240db4b8f"
Oct 31 00:52:26.466646 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.466626    4924 scope.go:122] "RemoveContainer" containerID="2914c451bd5be163c9ff2830e2e0447fd6cf883425daf9a8a678c223cade5c61"
Oct 31 00:52:26.474971 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.474953    4924 scope.go:122] "RemoveContainer" containerID="7afa6c0977e64891cc1b1a2d5ccd9bec54798381c3756553c3d8eab471e1c733"
Oct 31 00:52:26.483279 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.483258    4924 scope.go:122] "RemoveContainer" containerID="c29441fd7d455eb8ab38793a0440c7b430e934c08a39ff2aabdfdd5fe1f58b05"
Oct 31 00:52:26.491616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.491594    4924 scope.go:122] "RemoveContainer" containerID="f7fff139103c70d22d0884bf652ee78b35c3df4d87ecc552fcfa7d98398e83b8"
Oct 31 00:52:26.500327 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:26.500304    4924 scope.go:122] "RemoveContainer" containerID="dbba89321e9b73943b74625fd28e02f2c7fe5beabf60fadfd30208ddd6a4b134"
Oct 31 00:52:27.852698 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:52:27.852658    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:52:30.037144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:30.037108    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"]
Oct 31 00:52:30.040349 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:30.040317    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ca9535c5-qpqgt"]
Oct 31 00:52:31.763417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:31.763379    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1e5244c-8ea2-454f-a1fe-ca4333aeba6a" path="/var/lib/kubelet/pods/b1e5244c-8ea2-454f-a1fe-ca4333aeba6a/volumes"
Oct 31 00:52:40.034708 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:40.034668    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"]
Oct 31 00:52:40.038024 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:40.037998    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b7af06a7-5hgmv"]
Oct 31 00:52:41.024104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:41.024068    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"]
Oct 31 00:52:41.027076 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:41.027052    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c37a93f0-jxgx2"]
Oct 31 00:52:41.763646 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:41.763613    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67f1d65d-b776-43cd-9c2e-c47ba2c85ab4" path="/var/lib/kubelet/pods/67f1d65d-b776-43cd-9c2e-c47ba2c85ab4/volumes"
Oct 31 00:52:41.764092 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:41.764076    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8f7a776-684d-4887-8b8b-38d19f5d9fc5" path="/var/lib/kubelet/pods/c8f7a776-684d-4887-8b8b-38d19f5d9fc5/volumes"
Oct 31 00:52:46.022529 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:46.022484    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"]
Oct 31 00:52:46.025865 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:46.025835    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-36ccc54c-ws9wf"]
Oct 31 00:52:47.764487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:47.764451    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="826c98b9-62fc-4e93-b0ae-6b0ebfcf0314" path="/var/lib/kubelet/pods/826c98b9-62fc-4e93-b0ae-6b0ebfcf0314/volumes"
Oct 31 00:52:53.023227 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:53.023179    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"]
Oct 31 00:52:53.026138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:53.026112    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dc74f563-gczhp"]
Oct 31 00:52:53.765276 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:52:53.765239    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1297a896-1e24-469f-bc4c-87e1a44495cd" path="/var/lib/kubelet/pods/1297a896-1e24-469f-bc4c-87e1a44495cd/volumes"
Oct 31 00:52:57.854839 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:52:57.854800    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:53:00.024361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:00.024321    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"]
Oct 31 00:53:00.027412 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:00.027383    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-852b427e-wn7wl"]
Oct 31 00:53:01.771123 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:01.771086    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5" path="/var/lib/kubelet/pods/d6fb9dd6-b8c4-42df-be1b-4f01e70ea0e5/volumes"
Oct 31 00:53:04.023701 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:04.023669    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"]
Oct 31 00:53:04.027052 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:04.027022    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dcfa16ae-vqlwc"]
Oct 31 00:53:05.763724 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:05.763688    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47772251-ea7e-4c8d-a541-0b81d7015d2d" path="/var/lib/kubelet/pods/47772251-ea7e-4c8d-a541-0b81d7015d2d/volumes"
Oct 31 00:53:09.034644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:09.034601    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"]
Oct 31 00:53:09.038058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:09.038030    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"]
Oct 31 00:53:09.040698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:09.040686    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b66834d-ms2qk"]
Oct 31 00:53:09.043399 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:09.043373    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a2259cd-x2dfm"]
Oct 31 00:53:09.764035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:09.764005    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef3ebc53-8a7d-41c6-8117-45372d57513c" path="/var/lib/kubelet/pods/ef3ebc53-8a7d-41c6-8117-45372d57513c/volumes"
Oct 31 00:53:09.764453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:09.764430    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb0c9073-46bd-4718-8d47-84a00663c82d" path="/var/lib/kubelet/pods/fb0c9073-46bd-4718-8d47-84a00663c82d/volumes"
Oct 31 00:53:12.024548 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:12.024494    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"]
Oct 31 00:53:12.027530 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:12.027498    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9a3382d-vvvnk"]
Oct 31 00:53:13.764328 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:13.764285    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b1e0dfe-ec09-45ce-922d-6a21740f3c5b" path="/var/lib/kubelet/pods/2b1e0dfe-ec09-45ce-922d-6a21740f3c5b/volumes"
Oct 31 00:53:16.024436 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:16.024395    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"]
Oct 31 00:53:16.027808 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:16.027771    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"]
Oct 31 00:53:16.030549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:16.030519    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e19fbd-98wls"]
Oct 31 00:53:16.034243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:16.034212    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d346c63e-76rpw"]
Oct 31 00:53:17.763447 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:17.763410    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="758a34a0-6677-4b2a-ae80-0d9171133763" path="/var/lib/kubelet/pods/758a34a0-6677-4b2a-ae80-0d9171133763/volumes"
Oct 31 00:53:17.763863 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:17.763855    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="feea9704-4418-474f-afa3-16a00f63233c" path="/var/lib/kubelet/pods/feea9704-4418-474f-afa3-16a00f63233c/volumes"
Oct 31 00:53:26.359253 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.359209    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:53:26.408057 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.408025    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:53:26.417571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.417540    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:53:26.671834 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.671730    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:53:26.680254 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.680222    4924 scope.go:122] "RemoveContainer" containerID="7932201f8142d9a2c01e405a4afb23fbcfaa48ff05b50da16c1a10e7ff15ac56"
Oct 31 00:53:26.840770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.829328    4924 scope.go:122] "RemoveContainer" containerID="8fc0db991dfd2e7e7ba092966f1fd5785876ef419eab5f8f86405d2589c07e9a"
Oct 31 00:53:26.962315 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.962179    4924 scope.go:122] "RemoveContainer" containerID="b2b25cfe506c4edfe0e4ddb4a01e703678ef8a357793c39772df61f8b5f40caf"
Oct 31 00:53:26.983815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:26.983789    4924 scope.go:122] "RemoveContainer" containerID="176aa9d00a4b90d2a25826515a72252c12aba9c2fd33fb9ef3910506b7b024b5"
Oct 31 00:53:27.046770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:27.029290    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"]
Oct 31 00:53:27.052919 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:27.052849    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c960dd27-2w6zg"]
Oct 31 00:53:27.071792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:27.068701    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"]
Oct 31 00:53:27.106044 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:27.105609    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-486c3564-p2kbv"]
Oct 31 00:53:27.765355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:27.765317    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="230f3f79-4978-42c9-89a9-f32eaa09257e" path="/var/lib/kubelet/pods/230f3f79-4978-42c9-89a9-f32eaa09257e/volumes"
Oct 31 00:53:27.765893 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:27.765825    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="461484c0-90e3-4f1a-959a-847f1e9e935b" path="/var/lib/kubelet/pods/461484c0-90e3-4f1a-959a-847f1e9e935b/volumes"
Oct 31 00:53:27.854846 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:53:27.854809    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:53:31.022781 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:31.022728    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"]
Oct 31 00:53:31.026629 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:31.026592    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da902600-zxwdt"]
Oct 31 00:53:31.763187 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:31.763091    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8" path="/var/lib/kubelet/pods/1b7eedf5-22f2-4482-99ee-0f5a50ec0ff8/volumes"
Oct 31 00:53:36.030150 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:36.030051    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"]
Oct 31 00:53:36.032832 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:36.032802    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5460bdd0-nbt92"]
Oct 31 00:53:37.764243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:37.764205    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c0ae94c-fbfe-4259-8b61-5dd9e1b10398" path="/var/lib/kubelet/pods/3c0ae94c-fbfe-4259-8b61-5dd9e1b10398/volumes"
Oct 31 00:53:44.030600 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:44.030521    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"]
Oct 31 00:53:44.033949 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:44.033920    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddaa97b6-5g6fc"]
Oct 31 00:53:45.765098 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:45.764979    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7945106c-574a-4b09-8d93-4366e9d0aa07" path="/var/lib/kubelet/pods/7945106c-574a-4b09-8d93-4366e9d0aa07/volumes"
Oct 31 00:53:53.028107 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:53.028049    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"]
Oct 31 00:53:53.032439 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:53.032393    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-faed4c37-8wd46"]
Oct 31 00:53:53.848770 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:53.841146    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e35c113-5708-417e-988e-537f2c3d8558" path="/var/lib/kubelet/pods/5e35c113-5708-417e-988e-537f2c3d8558/volumes"
Oct 31 00:53:54.064833 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:54.064793    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"]
Oct 31 00:53:54.071628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:54.071583    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dabf9fd9-dgtsx"]
Oct 31 00:53:55.146908 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:55.146862    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-9c7vk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:53:55.147537 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:55.147500    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-9c7vk" podUID="9495ab3a-0b84-4854-bc5e-30d46c9f9040" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:53:55.296892 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:55.296844    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-pmzpl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:53:55.297203 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:55.297174    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-pmzpl" podUID="fc4b4307-477d-44cd-a66a-66eb109867be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:53:55.764063 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:55.764023    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1" path="/var/lib/kubelet/pods/f3a2c7e2-3b51-4c2f-bfce-0ba7183808a1/volumes"
Oct 31 00:53:57.852328 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:53:57.852286    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:53:59.025191 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:59.025153    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"]
Oct 31 00:53:59.029644 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:59.029614    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1657762c-6sk4x"]
Oct 31 00:53:59.762901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:53:59.762867    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a18a7547-2609-4876-895d-d378d5582f32" path="/var/lib/kubelet/pods/a18a7547-2609-4876-895d-d378d5582f32/volumes"
Oct 31 00:54:24.286670 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.286618    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"]
Oct 31 00:54:24.287238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.286900    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx" podUID="ef1215c9-b847-4ddd-81b0-599213179324" containerName="discovery" containerID="cri-o://b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1" gracePeriod=30
Oct 31 00:54:24.548792 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.548698    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:54:24.661075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661035    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.661274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661095    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.661274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661138    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.661274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661157    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.661274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661205    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.661274 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661277    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.661585 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.661310    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap\") pod \"ef1215c9-b847-4ddd-81b0-599213179324\" (UID: \"ef1215c9-b847-4ddd-81b0-599213179324\") "
Oct 31 00:54:24.662089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.662057    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:54:24.663584 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.663551    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:54:24.663701 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.663674    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "kube-api-access-th5x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:54:24.663908 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.663882    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:54:24.664003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.663911    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:54:24.664003 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.663972    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:54:24.664079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.664006    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig" pod "ef1215c9-b847-4ddd-81b0-599213179324" (UID: "ef1215c9-b847-4ddd-81b0-599213179324"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:54:24.762475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762433    4924 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-cacerts\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.762475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762472    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.762697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762489    4924 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-ca-configmap\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.762697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762505    4924 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef1215c9-b847-4ddd-81b0-599213179324-local-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.762697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762520    4924 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-csr-dns-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.762697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762537    4924 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef1215c9-b847-4ddd-81b0-599213179324-istio-kubeconfig\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.762697 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.762551    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th5x6\" (UniqueName: \"kubernetes.io/projected/ef1215c9-b847-4ddd-81b0-599213179324-kube-api-access-th5x6\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:24.779020 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.778981    4924 generic.go:356] "Generic (PLEG): container finished" podID="ef1215c9-b847-4ddd-81b0-599213179324" containerID="b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1" exitCode=0
Oct 31 00:54:24.779171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.779034    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"
Oct 31 00:54:24.779171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.779067    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx" event={"ID":"ef1215c9-b847-4ddd-81b0-599213179324","Type":"ContainerDied","Data":"b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1"}
Oct 31 00:54:24.779171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.779119    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx" event={"ID":"ef1215c9-b847-4ddd-81b0-599213179324","Type":"ContainerDied","Data":"11da07e5b34cbee317621fab823c3731905f0ed8fb27c95226bccf47a0f75c40"}
Oct 31 00:54:24.779171 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.779144    4924 scope.go:122] "RemoveContainer" containerID="b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1"
Oct 31 00:54:24.790272 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.790245    4924 scope.go:122] "RemoveContainer" containerID="b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1"
Oct 31 00:54:24.790561 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:54:24.790542    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1\": container with ID starting with b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1 not found: ID does not exist" containerID="b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1"
Oct 31 00:54:24.790620 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.790568    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1"} err="failed to get container status \"b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1\": rpc error: code = NotFound desc = could not find container \"b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1\": container with ID starting with b591dbb8840f3d7d1ee31fa330a5f835f86c2adb908093dd9573a9bb14435bd1 not found: ID does not exist"
Oct 31 00:54:24.802782 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.802676    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"]
Oct 31 00:54:24.806648 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:24.806617    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qk8gx"]
Oct 31 00:54:25.700361 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.700315    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"]
Oct 31 00:54:25.700985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.700968    4924 cpu_manager.go:414] "containerMap: removing container" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy"
Oct 31 00:54:25.701035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.700988    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy"
Oct 31 00:54:25.701035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701001    4924 cpu_manager.go:414] "containerMap: removing container" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerName="payload-processing"
Oct 31 00:54:25.701035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701009    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerName="payload-processing"
Oct 31 00:54:25.701035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701022    4924 cpu_manager.go:414] "containerMap: removing container" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy"
Oct 31 00:54:25.701035 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701031    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701050    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ef1215c9-b847-4ddd-81b0-599213179324" containerName="discovery"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701058    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef1215c9-b847-4ddd-81b0-599213179324" containerName="discovery"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701072    4924 cpu_manager.go:414] "containerMap: removing container" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701079    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701091    4924 cpu_manager.go:414] "containerMap: removing container" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerName="payload-pre-processing"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701098    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerName="payload-pre-processing"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701109    4924 cpu_manager.go:414] "containerMap: removing container" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701116    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701136    4924 cpu_manager.go:414] "containerMap: removing container" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701143    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701158    4924 cpu_manager.go:414] "containerMap: removing container" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701165    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701176    4924 cpu_manager.go:414] "containerMap: removing container" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701183    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy"
Oct 31 00:54:25.701197 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701196    4924 cpu_manager.go:414] "containerMap: removing container" podUID="e9f27690-bb8e-4159-b918-84baff256f09" containerName="payload-processing"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701204    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9f27690-bb8e-4159-b918-84baff256f09" containerName="payload-processing"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701217    4924 cpu_manager.go:414] "containerMap: removing container" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701224    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701233    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701240    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701250    4924 cpu_manager.go:414] "containerMap: removing container" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701257    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701270    4924 cpu_manager.go:414] "containerMap: removing container" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701277    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701292    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701298    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701307    4924 cpu_manager.go:414] "containerMap: removing container" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701314    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701336    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701342    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701351    4924 cpu_manager.go:414] "containerMap: removing container" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701358    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701367    4924 cpu_manager.go:414] "containerMap: removing container" podUID="a18a7547-2609-4876-895d-d378d5582f32" containerName="revoke-keys"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701374    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a18a7547-2609-4876-895d-d378d5582f32" containerName="revoke-keys"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701390    4924 cpu_manager.go:414] "containerMap: removing container" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701397    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701495    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e29f410-c8ad-44ca-98a2-df59935478ef" containerName="main"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701510    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6643d43-f92c-4820-b206-cd6e78a7c3b5" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701522    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="23cea57f-ad09-4f8d-9638-b2b1b7cce1b7" containerName="payload-processing"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701530    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="847e6d33-4e12-49e7-96c3-7f2bc41300fc" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701540    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef1215c9-b847-4ddd-81b0-599213179324" containerName="discovery"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701551    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701560    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb5f27fd-5a43-49b3-8644-8096e11165ed" containerName="payload-pre-processing"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701569    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="914a5656-be09-46ed-8413-633981153991" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701578    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e0ed6a7-2eb3-42a8-9de3-a6ca8cf2a85d" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701589    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="88b08800-ec7f-4d04-85a2-ef12a01a4c10" containerName="main"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701598    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="b80533bd-0a09-4909-9ad8-d6ab1b0da4fb" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701605    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9f27690-bb8e-4159-b918-84baff256f09" containerName="payload-processing"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701614    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701622    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d25a8ae5-a217-4859-ac56-9ec49996c29a" containerName="istio-proxy"
Oct 31 00:54:25.701623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701631    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f51538d-1ba0-48aa-8dda-a0c7bcc62f3b" containerName="istio-proxy"
Oct 31 00:54:25.702756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701639    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2b570d4-6c8e-4454-a84a-37c4c4118471" containerName="istio-proxy"
Oct 31 00:54:25.702756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701648    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac16d450-266c-4cce-a250-0725443133c4" containerName="main"
Oct 31 00:54:25.702756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701667    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="8677e702-1ca1-469c-bdd7-76e4f1512c9c" containerName="istio-proxy"
Oct 31 00:54:25.702756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701678    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ae76819-9bd6-4f27-9fa9-b63ff98440fb" containerName="istio-proxy"
Oct 31 00:54:25.702756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701686    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f4b1e70-835c-4f70-a519-46ff69973f9f" containerName="istio-proxy"
Oct 31 00:54:25.702756 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.701696    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="a18a7547-2609-4876-895d-d378d5582f32" containerName="revoke-keys"
Oct 31 00:54:25.706975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.706953    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.710885 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.710858    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:54:25.711093 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.711076    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:54:25.712475 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.712448    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"]
Oct 31 00:54:25.771721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.771687    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef1215c9-b847-4ddd-81b0-599213179324" path="/var/lib/kubelet/pods/ef1215c9-b847-4ddd-81b0-599213179324/volumes"
Oct 31 00:54:25.872837 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.872802    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.872998 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.872922    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmklm\" (UniqueName: \"kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.873119 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.873096    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.974165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.974062    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.974344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.974234    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.974344 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.974288    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmklm\" (UniqueName: \"kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.974926 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.974907    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.976531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.976510    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:25.983009 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:25.982981    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmklm\" (UniqueName: \"kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm\") pod \"maas-api-revoke-sub-3d4156f1-t8qx2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:26.025104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:26.025066    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:26.307345 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:26.307307    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"]
Oct 31 00:54:26.307992 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:54:26.307959    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3dc9e03_2a00_4dbc_81eb_5dd8f73252c2.slice/crio-ab88751418688e4ca85de6cd231f70e9f490c711c1ceab77d931b130ddb73e2a WatchSource:0}: Error finding container ab88751418688e4ca85de6cd231f70e9f490c711c1ceab77d931b130ddb73e2a: Status 404 returned error can't find the container with id ab88751418688e4ca85de6cd231f70e9f490c711c1ceab77d931b130ddb73e2a
Oct 31 00:54:26.794552 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:26.794515    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2" event={"ID":"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2","Type":"ContainerStarted","Data":"109b180f7fbe5ab73cf6611f705455abb49373925d69cb775fe0bb198c067435"}
Oct 31 00:54:26.794941 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:26.794568    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2" event={"ID":"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2","Type":"ContainerStarted","Data":"ab88751418688e4ca85de6cd231f70e9f490c711c1ceab77d931b130ddb73e2a"}
Oct 31 00:54:26.813049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:26.812994    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2" podStartSLOduration=1.812979746 podStartE2EDuration="1.812979746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:54:26.810888016 +0000 UTC m=+1267.181671805" watchObservedRunningTime="2026-10-31 00:54:26.812979746 +0000 UTC m=+1267.183763535"
Oct 31 00:54:27.154355 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.154317    4924 scope.go:122] "RemoveContainer" containerID="ee28f9b146b3f0acdbc1cf87c20aa1313e96e1780625293608f08909baf265b1"
Oct 31 00:54:27.164331 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.164307    4924 scope.go:122] "RemoveContainer" containerID="d4d8253d21a9d9320fa69b5c9b7f1f76598d9b0b3e4de57d4b56524320e7650a"
Oct 31 00:54:27.173546 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.173516    4924 scope.go:122] "RemoveContainer" containerID="dc9fb93318b9a720ad8e1533505430a506a8357060b9591b1e3a57a1d37dbaa4"
Oct 31 00:54:27.182461 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.182438    4924 scope.go:122] "RemoveContainer" containerID="7ae2dac79bb566488261765ae5bbfbb8e6151d7b5c4d87780dc6cf3fb011e649"
Oct 31 00:54:27.191113 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.191092    4924 scope.go:122] "RemoveContainer" containerID="60b19aa8419b3938f72bc4fab753ca6d7c1a4b6fe41175412caf59f5ab1421ff"
Oct 31 00:54:27.230482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.230461    4924 scope.go:122] "RemoveContainer" containerID="bec7018d6b4320ddc4ed4afe029a93ff1aa70b07122f8853f892d39b1c667dbe"
Oct 31 00:54:27.239549 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.239526    4924 scope.go:122] "RemoveContainer" containerID="6866c1e9b88fdc696b0551393ba95fb417cf8a7df5e55481c05f40d8a93abf0a"
Oct 31 00:54:27.248632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.248608    4924 scope.go:122] "RemoveContainer" containerID="c9688770585fee0460b3701db84992379ef384c83b9407ac4204408652ce8f30"
Oct 31 00:54:27.257278 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.257258    4924 scope.go:122] "RemoveContainer" containerID="e822aa4c17c6bf62308fbb73e008c3cfad5748b47e0845f3552613a5334c92ff"
Oct 31 00:54:27.265851 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.265830    4924 scope.go:122] "RemoveContainer" containerID="e9ab42c7d8731b684fb652ee29487468709d785207d51626b0359369e7bc95e6"
Oct 31 00:54:27.274333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.274313    4924 scope.go:122] "RemoveContainer" containerID="f06fc8c0f355ef9f7183a1dbeb8ca8711e7d31703a5fe75f8b8fb51c60f7c6b2"
Oct 31 00:54:27.282577 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.282558    4924 scope.go:122] "RemoveContainer" containerID="ba2c1b1afc8162f99e9f9a6a75b2e2cea373cb9b6c5180289943b75fe69d2959"
Oct 31 00:54:27.290774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.290734    4924 scope.go:122] "RemoveContainer" containerID="d3b7585bb72246133090a2d80a2bff4392b1582727a9bf519270641d2b06642d"
Oct 31 00:54:27.299469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.299451    4924 scope.go:122] "RemoveContainer" containerID="a8d0601f4993f3e0b41e67db2c88212b75e825da7c742cfe390eb1d0794a8f3d"
Oct 31 00:54:27.801759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.801642    4924 generic.go:356] "Generic (PLEG): container finished" podID="d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" containerID="109b180f7fbe5ab73cf6611f705455abb49373925d69cb775fe0bb198c067435" exitCode=0
Oct 31 00:54:27.801759 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:27.801725    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2" event={"ID":"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2","Type":"ContainerDied","Data":"109b180f7fbe5ab73cf6611f705455abb49373925d69cb775fe0bb198c067435"}
Oct 31 00:54:27.854145 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:54:27.854106    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:54:28.958971 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:28.958933    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:29.115579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.115486    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm\" (UniqueName: \"kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm\") pod \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") "
Oct 31 00:54:29.115579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.115579    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca\") pod \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") "
Oct 31 00:54:29.115913 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.115694    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca\") pod \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\" (UID: \"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2\") "
Oct 31 00:54:29.116102 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.116069    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca" pod "d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" (UID: "d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:54:29.118043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.118012    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca" pod "d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" (UID: "d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:54:29.118043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.118025    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm" pod "d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" (UID: "d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2"). InnerVolumeSpecName "kube-api-access-qmklm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:54:29.217130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.217084    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:29.217130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.217124    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:29.217130 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.217136    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmklm\" (UniqueName: \"kubernetes.io/projected/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2-kube-api-access-qmklm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:29.815694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.815647    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2" event={"ID":"d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2","Type":"ContainerDied","Data":"ab88751418688e4ca85de6cd231f70e9f490c711c1ceab77d931b130ddb73e2a"}
Oct 31 00:54:29.815874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.815729    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab88751418688e4ca85de6cd231f70e9f490c711c1ceab77d931b130ddb73e2a"
Oct 31 00:54:29.815874 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:29.815663    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"
Oct 31 00:54:45.475055 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.475019    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"]
Oct 31 00:54:45.475649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.475617    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" containerName="revoke-keys"
Oct 31 00:54:45.475649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.475639    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" containerName="revoke-keys"
Oct 31 00:54:45.475858 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.475731    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" containerName="revoke-keys"
Oct 31 00:54:45.479128 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.479107    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.498026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.497995    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"]
Oct 31 00:54:45.525950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.525907    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.525956    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526055    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526117    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526324 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526158    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526302    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526353    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526469 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526430    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb8w4\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.526516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.526477    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.537930 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.537895    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-fb32d0-openshift-default-dockercfg-tgw76\""
Oct 31 00:54:45.627290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627250    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627291    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627316    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627349    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627377    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627445    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627488    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627531    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb8w4\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.627567 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627565    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.628019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627892    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.628019 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.627936    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.628138 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.628030    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.628351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.628323    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.628478 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.628456    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.629849 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.629823    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.630071 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.630052    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.635429 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.635395    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.636086 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.636064    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb8w4\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4\") pod \"e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") " pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:45.793196 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:45.793160    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:46.080789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.080757    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"]
Oct 31 00:54:46.082009 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:54:46.081973    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34d3205_870b_476c_b376_964e6a79b049.slice/crio-0c0e6876a3cace33bac374f08ae6309bcb69c9788455d7983b5e3e09028198cd WatchSource:0}: Error finding container 0c0e6876a3cace33bac374f08ae6309bcb69c9788455d7983b5e3e09028198cd: Status 404 returned error can't find the container with id 0c0e6876a3cace33bac374f08ae6309bcb69c9788455d7983b5e3e09028198cd
Oct 31 00:54:46.085578 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.085537    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:54:46.085666 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.085619    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:54:46.085705 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.085665    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:54:46.964018 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.963972    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" event={"ID":"d34d3205-870b-476c-b376-964e6a79b049","Type":"ContainerStarted","Data":"ff22f0e78bccc2397acbb8560977f497e1d61f4500ed5ec6730cab8fd1845dce"}
Oct 31 00:54:46.964456 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.964039    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" event={"ID":"d34d3205-870b-476c-b376-964e6a79b049","Type":"ContainerStarted","Data":"0c0e6876a3cace33bac374f08ae6309bcb69c9788455d7983b5e3e09028198cd"}
Oct 31 00:54:46.987901 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:46.987849    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podStartSLOduration=1.987835266 podStartE2EDuration="1.987835266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:54:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:54:46.986564379 +0000 UTC m=+1287.357348172" watchObservedRunningTime="2026-10-31 00:54:46.987835266 +0000 UTC m=+1287.358619055"
Oct 31 00:54:47.794299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:47.794260    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:47.799623 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:47.799590    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:47.970038 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:47.970000    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:47.971104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:47.971081    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:54:51.000216 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.000182    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"]
Oct 31 00:54:51.004370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.004343    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.013628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.013599    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-fb32d0\""
Oct 31 00:54:51.018686 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.018662    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"]
Oct 31 00:54:51.083495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.083457    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2szs9\" (UniqueName: \"kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9\") pod \"payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.083495 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.083493    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.184736 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.184689    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2szs9\" (UniqueName: \"kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9\") pod \"payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.184925 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.184761    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.185740 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.185709    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.193131 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.193106    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2szs9\" (UniqueName: \"kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9\") pod \"payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.321638 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.321596    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:51.633667 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.633622    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"]
Oct 31 00:54:51.636149 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:54:51.636113    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269cc154_2460_4b1f_b4ac_80478f6fad8c.slice/crio-101c783d36d8b2a4088800dd3b1d850b4ef94b1062de0a25ecd947a3eb203b46 WatchSource:0}: Error finding container 101c783d36d8b2a4088800dd3b1d850b4ef94b1062de0a25ecd947a3eb203b46: Status 404 returned error can't find the container with id 101c783d36d8b2a4088800dd3b1d850b4ef94b1062de0a25ecd947a3eb203b46
Oct 31 00:54:51.994680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.994584    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" event={"ID":"269cc154-2460-4b1f-b4ac-80478f6fad8c","Type":"ContainerStarted","Data":"7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632"}
Oct 31 00:54:51.994875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.994679    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" event={"ID":"269cc154-2460-4b1f-b4ac-80478f6fad8c","Type":"ContainerStarted","Data":"101c783d36d8b2a4088800dd3b1d850b4ef94b1062de0a25ecd947a3eb203b46"}
Oct 31 00:54:51.994875 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:51.994711    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:54:52.037614 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:54:52.037560    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" podStartSLOduration=2.037546459 podStartE2EDuration="2.037546459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:54:52.035919687 +0000 UTC m=+1292.406703476" watchObservedRunningTime="2026-10-31 00:54:52.037546459 +0000 UTC m=+1292.408330248"
Oct 31 00:54:57.853795 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:54:57.853733    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:55:02.971606 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:02.971524    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 31 00:55:02.972060 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:02.971608    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 31 00:55:03.004118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.004087    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:55:03.096988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.096883    4924 generic.go:356] "Generic (PLEG): container finished" podID="d34d3205-870b-476c-b376-964e6a79b049" containerID="ff22f0e78bccc2397acbb8560977f497e1d61f4500ed5ec6730cab8fd1845dce" exitCode=0
Oct 31 00:55:03.096988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.096958    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" event={"ID":"d34d3205-870b-476c-b376-964e6a79b049","Type":"ContainerDied","Data":"ff22f0e78bccc2397acbb8560977f497e1d61f4500ed5ec6730cab8fd1845dce"}
Oct 31 00:55:03.097516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.097488    4924 scope.go:122] "RemoveContainer" containerID="ff22f0e78bccc2397acbb8560977f497e1d61f4500ed5ec6730cab8fd1845dce"
Oct 31 00:55:03.099139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.099098    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:55:03.099246 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.099201    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:55:03.099286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:03.099260    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:55:04.126652 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:04.126615    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" event={"ID":"d34d3205-870b-476c-b376-964e6a79b049","Type":"ContainerStarted","Data":"60992f2f477110423e9516a528fa8a7785d3c62dd014686f5f685379bf3561e8"}
Oct 31 00:55:04.806162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:04.806122    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:55:04.806544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:04.806458    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 31 00:55:04.806713 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:04.806691    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 31 00:55:05.798219 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:05.798176    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 31 00:55:05.811891 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:05.811834    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 31 00:55:06.801772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:06.797179    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 31 00:55:06.801772 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:06.797284    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 31 00:55:07.796186 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:07.796143    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 31 00:55:07.796501 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:07.796471    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 31 00:55:08.796790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:08.794458    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 31 00:55:08.796790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:08.794526    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 31 00:55:09.798000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:09.797960    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:55:09.798464 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:09.798270    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:55:09.799165 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:09.799147    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:55:27.387079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.387044    4924 scope.go:122] "RemoveContainer" containerID="67a1ce7a4d5c000a8ef085cfd5f76b793c027211997b12c7a922a482219d5b48"
Oct 31 00:55:27.396371 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.396349    4924 scope.go:122] "RemoveContainer" containerID="1e8a546c76fb0388c99a2a897d816a6002191c3840a5ec649438513beed4d05c"
Oct 31 00:55:27.424329 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.422990    4924 scope.go:122] "RemoveContainer" containerID="bb9881a0b47edae0588ffb57c74795a9dfc197ee3557195b5893983619100e1f"
Oct 31 00:55:27.433579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.433529    4924 scope.go:122] "RemoveContainer" containerID="deba036cf0e6d4f92f4a7bc0b2f219c653c20f40dd25c5c43b9442031ffb2eb7"
Oct 31 00:55:27.442198 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.442174    4924 scope.go:122] "RemoveContainer" containerID="0e3263b05bb8a87a6a21b162dfd54304f606af9b71cbc229e933165d8f832ca4"
Oct 31 00:55:27.450480 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.450462    4924 scope.go:122] "RemoveContainer" containerID="5f85453d10876b497dce09427167cb40a674398ce7e1d2f195bbcae15d9c3863"
Oct 31 00:55:27.458621 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.458601    4924 scope.go:122] "RemoveContainer" containerID="a349585f0d067514993c5f00ad98e90ceb1032517f4ed25a2d0562374aaf9bac"
Oct 31 00:55:27.466551 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:27.466521    4924 scope.go:122] "RemoveContainer" containerID="ee1a8df48f3d8c1e5ddec52c98e0f4032dd462b7dc9ac4d7fac99772e405d041"
Oct 31 00:55:27.852186 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:55:27.852140    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:55:32.116196 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.116156    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"]
Oct 31 00:55:32.122376 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.122352    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.125866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.125821    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:55:32.125999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.125924    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:55:32.128292 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.128269    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"]
Oct 31 00:55:32.173170 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.173132    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.173318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.173191    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.173373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.173325    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cmsv\" (UniqueName: \"kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.274249 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.274209    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.274437 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.274289    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cmsv\" (UniqueName: \"kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.274437 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.274347    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.275069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.275048    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.276705 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.276681    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.282313 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.282282    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cmsv\" (UniqueName: \"kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv\") pod \"maas-api-revoke-sub-f9b0664a-s95q8\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.437366 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.437272    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:32.716980 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:32.716946    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"]
Oct 31 00:55:32.718676 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:55:32.718649    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c72824a_6c3c_49bc_998b_6d8830fb125a.slice/crio-a754c459db5cecc539bf1f8fa21992786bf429673190b0de1f03525794009594 WatchSource:0}: Error finding container a754c459db5cecc539bf1f8fa21992786bf429673190b0de1f03525794009594: Status 404 returned error can't find the container with id a754c459db5cecc539bf1f8fa21992786bf429673190b0de1f03525794009594
Oct 31 00:55:33.312058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:33.311995    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8" event={"ID":"1c72824a-6c3c-49bc-998b-6d8830fb125a","Type":"ContainerStarted","Data":"b030ead98baa4ae0faef0e4efd981c48746d49554ab9ab8a7261228d5132532e"}
Oct 31 00:55:33.312579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:33.312101    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8" event={"ID":"1c72824a-6c3c-49bc-998b-6d8830fb125a","Type":"ContainerStarted","Data":"a754c459db5cecc539bf1f8fa21992786bf429673190b0de1f03525794009594"}
Oct 31 00:55:33.340774 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:33.340401    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8" podStartSLOduration=1.340380541 podStartE2EDuration="1.340380541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:33.328036537 +0000 UTC m=+1333.698820329" watchObservedRunningTime="2026-10-31 00:55:33.340380541 +0000 UTC m=+1333.711164333"
Oct 31 00:55:34.321241 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:34.321200    4924 generic.go:356] "Generic (PLEG): container finished" podID="1c72824a-6c3c-49bc-998b-6d8830fb125a" containerID="b030ead98baa4ae0faef0e4efd981c48746d49554ab9ab8a7261228d5132532e" exitCode=0
Oct 31 00:55:34.321685 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:34.321268    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8" event={"ID":"1c72824a-6c3c-49bc-998b-6d8830fb125a","Type":"ContainerDied","Data":"b030ead98baa4ae0faef0e4efd981c48746d49554ab9ab8a7261228d5132532e"}
Oct 31 00:55:35.470569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.470541    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:35.511544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.511507    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca\") pod \"1c72824a-6c3c-49bc-998b-6d8830fb125a\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") "
Oct 31 00:55:35.511544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.511551    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv\" (UniqueName: \"kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv\") pod \"1c72824a-6c3c-49bc-998b-6d8830fb125a\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") "
Oct 31 00:55:35.511824 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.511667    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca\") pod \"1c72824a-6c3c-49bc-998b-6d8830fb125a\" (UID: \"1c72824a-6c3c-49bc-998b-6d8830fb125a\") "
Oct 31 00:55:35.511967 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.511937    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca" pod "1c72824a-6c3c-49bc-998b-6d8830fb125a" (UID: "1c72824a-6c3c-49bc-998b-6d8830fb125a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:35.513866 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.513833    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca" pod "1c72824a-6c3c-49bc-998b-6d8830fb125a" (UID: "1c72824a-6c3c-49bc-998b-6d8830fb125a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:55:35.514185 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.514157    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv" pod "1c72824a-6c3c-49bc-998b-6d8830fb125a" (UID: "1c72824a-6c3c-49bc-998b-6d8830fb125a"). InnerVolumeSpecName "kube-api-access-7cmsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:35.613094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.612994    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:35.613094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.613032    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c72824a-6c3c-49bc-998b-6d8830fb125a-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:35.613094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:35.613051    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cmsv\" (UniqueName: \"kubernetes.io/projected/1c72824a-6c3c-49bc-998b-6d8830fb125a-kube-api-access-7cmsv\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:36.333900 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:36.333869    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"
Oct 31 00:55:36.334122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:36.333910    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8" event={"ID":"1c72824a-6c3c-49bc-998b-6d8830fb125a","Type":"ContainerDied","Data":"a754c459db5cecc539bf1f8fa21992786bf429673190b0de1f03525794009594"}
Oct 31 00:55:36.334122 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:36.333955    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a754c459db5cecc539bf1f8fa21992786bf429673190b0de1f03525794009594"
Oct 31 00:55:43.182331 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.182287    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"]
Oct 31 00:55:43.182839 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.182824    4924 cpu_manager.go:414] "containerMap: removing container" podUID="1c72824a-6c3c-49bc-998b-6d8830fb125a" containerName="revoke-keys"
Oct 31 00:55:43.182907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.182841    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c72824a-6c3c-49bc-998b-6d8830fb125a" containerName="revoke-keys"
Oct 31 00:55:43.182955 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.182922    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c72824a-6c3c-49bc-998b-6d8830fb125a" containerName="revoke-keys"
Oct 31 00:55:43.186159 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.186138    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.191540 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.191515    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:55:43.192868 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.192848    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:55:43.195290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.194554    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"]
Oct 31 00:55:43.295869 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.295827    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.296134 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.296000    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.296231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.296131    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvwzb\" (UniqueName: \"kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.397078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.397041    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.397246 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.397113    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.397246 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.397158    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvwzb\" (UniqueName: \"kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.397710 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.397684    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.399491 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.399471    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.405007 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.404983    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvwzb\" (UniqueName: \"kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb\") pod \"maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.507281 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.507193    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:43.785906 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:43.785877    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"]
Oct 31 00:55:43.787241 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:55:43.787206    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44f2329_8d37_4961_9fae_6792ce1d5d92.slice/crio-689c2d2faed1e3d92fb8dace44f95076458ace1dd55b67404626c98d9227c6c0 WatchSource:0}: Error finding container 689c2d2faed1e3d92fb8dace44f95076458ace1dd55b67404626c98d9227c6c0: Status 404 returned error can't find the container with id 689c2d2faed1e3d92fb8dace44f95076458ace1dd55b67404626c98d9227c6c0
Oct 31 00:55:44.382295 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:44.382199    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz" event={"ID":"e44f2329-8d37-4961-9fae-6792ce1d5d92","Type":"ContainerStarted","Data":"eb891b9afb26ff4a4bfcbdd8c2762a6e35c60163136af52d3ec32445654ea4f3"}
Oct 31 00:55:44.382706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:44.382315    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz" event={"ID":"e44f2329-8d37-4961-9fae-6792ce1d5d92","Type":"ContainerStarted","Data":"689c2d2faed1e3d92fb8dace44f95076458ace1dd55b67404626c98d9227c6c0"}
Oct 31 00:55:44.400263 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:44.400201    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz" podStartSLOduration=1.4001842070000001 podStartE2EDuration="1.400184207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:44.397725793 +0000 UTC m=+1344.768509583" watchObservedRunningTime="2026-10-31 00:55:44.400184207 +0000 UTC m=+1344.770967995"
Oct 31 00:55:45.389387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:45.389353    4924 generic.go:356] "Generic (PLEG): container finished" podID="e44f2329-8d37-4961-9fae-6792ce1d5d92" containerID="eb891b9afb26ff4a4bfcbdd8c2762a6e35c60163136af52d3ec32445654ea4f3" exitCode=0
Oct 31 00:55:45.389823 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:45.389446    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz" event={"ID":"e44f2329-8d37-4961-9fae-6792ce1d5d92","Type":"ContainerDied","Data":"eb891b9afb26ff4a4bfcbdd8c2762a6e35c60163136af52d3ec32445654ea4f3"}
Oct 31 00:55:46.537168 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.537143    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:46.630643 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.630600    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca\") pod \"e44f2329-8d37-4961-9fae-6792ce1d5d92\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") "
Oct 31 00:55:46.630854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.630689    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb\" (UniqueName: \"kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb\") pod \"e44f2329-8d37-4961-9fae-6792ce1d5d92\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") "
Oct 31 00:55:46.630854 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.630781    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca\") pod \"e44f2329-8d37-4961-9fae-6792ce1d5d92\" (UID: \"e44f2329-8d37-4961-9fae-6792ce1d5d92\") "
Oct 31 00:55:46.631005 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.630981    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca" pod "e44f2329-8d37-4961-9fae-6792ce1d5d92" (UID: "e44f2329-8d37-4961-9fae-6792ce1d5d92"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:46.631144 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.631128    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:46.633089 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.633055    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca" pod "e44f2329-8d37-4961-9fae-6792ce1d5d92" (UID: "e44f2329-8d37-4961-9fae-6792ce1d5d92"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:55:46.633213 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.633092    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb" pod "e44f2329-8d37-4961-9fae-6792ce1d5d92" (UID: "e44f2329-8d37-4961-9fae-6792ce1d5d92"). InnerVolumeSpecName "kube-api-access-lvwzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:46.732639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.732551    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvwzb\" (UniqueName: \"kubernetes.io/projected/e44f2329-8d37-4961-9fae-6792ce1d5d92-kube-api-access-lvwzb\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:46.732639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:46.732583    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e44f2329-8d37-4961-9fae-6792ce1d5d92-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:47.403969 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:47.403936    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"
Oct 31 00:55:47.404183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:47.403939    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz" event={"ID":"e44f2329-8d37-4961-9fae-6792ce1d5d92","Type":"ContainerDied","Data":"689c2d2faed1e3d92fb8dace44f95076458ace1dd55b67404626c98d9227c6c0"}
Oct 31 00:55:47.404183 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:47.404054    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="689c2d2faed1e3d92fb8dace44f95076458ace1dd55b67404626c98d9227c6c0"
Oct 31 00:55:53.321543 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:53.321506    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"]
Oct 31 00:55:53.867814 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:53.867779    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"]
Oct 31 00:55:53.870193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:53.870168    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-fb32d0-vh4rz"]
Oct 31 00:55:55.789307 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:55.789263    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e44f2329-8d37-4961-9fae-6792ce1d5d92" path="/var/lib/kubelet/pods/e44f2329-8d37-4961-9fae-6792ce1d5d92/volumes"
Oct 31 00:55:57.852302 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:55:57.852263    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:55:58.322082 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:55:58.322025    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerName="payload-processing" containerID="cri-o://7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632" gracePeriod=25
Oct 31 00:56:00.381845 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:00.381803    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"]
Oct 31 00:56:00.382488 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:00.382085    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy" containerID="cri-o://60992f2f477110423e9516a528fa8a7785d3c62dd014686f5f685379bf3561e8" gracePeriod=30
Oct 31 00:56:01.659975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.659935    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"]
Oct 31 00:56:01.664789 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.664741    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-t8qx2"]
Oct 31 00:56:01.679985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.679955    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"]
Oct 31 00:56:01.680440 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.680427    4924 cpu_manager.go:414] "containerMap: removing container" podUID="e44f2329-8d37-4961-9fae-6792ce1d5d92" containerName="revoke-keys"
Oct 31 00:56:01.680483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.680441    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e44f2329-8d37-4961-9fae-6792ce1d5d92" containerName="revoke-keys"
Oct 31 00:56:01.680518 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.680514    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="e44f2329-8d37-4961-9fae-6792ce1d5d92" containerName="revoke-keys"
Oct 31 00:56:01.685026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.685007    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.688669 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.688644    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:56:01.689336 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.689312    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:56:01.691843 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.691812    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"]
Oct 31 00:56:01.724419 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.724379    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.724615 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.724522    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbwwn\" (UniqueName: \"kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.724615 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.724583    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.764001 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.763962    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2" path="/var/lib/kubelet/pods/d3dc9e03-2a00-4dbc-81eb-5dd8f73252c2/volumes"
Oct 31 00:56:01.825728 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.825689    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.825910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.825806    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbwwn\" (UniqueName: \"kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.825910 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.825856    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.826525 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.826499    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.828238 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.828214    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:01.834075 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:01.834050    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbwwn\" (UniqueName: \"kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn\") pod \"maas-api-revoke-sub-3d4156f1-rb7gb\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:02.003831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:02.003725    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:02.284111 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:02.284080    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"]
Oct 31 00:56:02.285971 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:56:02.285944    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aadf8a4_a548_4bda_98ce_674397a4c95e.slice/crio-ad99e5bad01d1c0849cddec599b739aaee40b0559d66ab0bdf04fd9910eb461e WatchSource:0}: Error finding container ad99e5bad01d1c0849cddec599b739aaee40b0559d66ab0bdf04fd9910eb461e: Status 404 returned error can't find the container with id ad99e5bad01d1c0849cddec599b739aaee40b0559d66ab0bdf04fd9910eb461e
Oct 31 00:56:02.530134 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:02.530100    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb" event={"ID":"4aadf8a4-a548-4bda-98ce-674397a4c95e","Type":"ContainerStarted","Data":"ad99e5bad01d1c0849cddec599b739aaee40b0559d66ab0bdf04fd9910eb461e"}
Oct 31 00:56:03.002267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:03.002179    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:56:03.537680 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:03.537637    4924 generic.go:356] "Generic (PLEG): container finished" podID="4aadf8a4-a548-4bda-98ce-674397a4c95e" containerID="9ad1c3343a6733ffc2dfa105be1401b615016da42544e36a82cdc6d258e641a9" exitCode=0
Oct 31 00:56:03.537898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:03.537694    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb" event={"ID":"4aadf8a4-a548-4bda-98ce-674397a4c95e","Type":"ContainerDied","Data":"9ad1c3343a6733ffc2dfa105be1401b615016da42544e36a82cdc6d258e641a9"}
Oct 31 00:56:04.683628 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.683604    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:04.753473 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.753440    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca\") pod \"4aadf8a4-a548-4bda-98ce-674397a4c95e\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") "
Oct 31 00:56:04.753807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.753506    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca\") pod \"4aadf8a4-a548-4bda-98ce-674397a4c95e\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") "
Oct 31 00:56:04.753807 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.753572    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn\" (UniqueName: \"kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn\") pod \"4aadf8a4-a548-4bda-98ce-674397a4c95e\" (UID: \"4aadf8a4-a548-4bda-98ce-674397a4c95e\") "
Oct 31 00:56:04.753950 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.753870    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca" pod "4aadf8a4-a548-4bda-98ce-674397a4c95e" (UID: "4aadf8a4-a548-4bda-98ce-674397a4c95e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:04.755699 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.755675    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn" pod "4aadf8a4-a548-4bda-98ce-674397a4c95e" (UID: "4aadf8a4-a548-4bda-98ce-674397a4c95e"). InnerVolumeSpecName "kube-api-access-sbwwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:04.755978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.755961    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca" pod "4aadf8a4-a548-4bda-98ce-674397a4c95e" (UID: "4aadf8a4-a548-4bda-98ce-674397a4c95e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:56:04.855148 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.855068    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:04.855148 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.855097    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbwwn\" (UniqueName: \"kubernetes.io/projected/4aadf8a4-a548-4bda-98ce-674397a4c95e-kube-api-access-sbwwn\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:04.855148 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:04.855107    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4aadf8a4-a548-4bda-98ce-674397a4c95e-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.550962 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.550932    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"
Oct 31 00:56:05.551149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.550975    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb" event={"ID":"4aadf8a4-a548-4bda-98ce-674397a4c95e","Type":"ContainerDied","Data":"ad99e5bad01d1c0849cddec599b739aaee40b0559d66ab0bdf04fd9910eb461e"}
Oct 31 00:56:05.551149 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.551022    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad99e5bad01d1c0849cddec599b739aaee40b0559d66ab0bdf04fd9910eb461e"
Oct 31 00:56:05.553333 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.553304    4924 generic.go:356] "Generic (PLEG): container finished" podID="d34d3205-870b-476c-b376-964e6a79b049" containerID="60992f2f477110423e9516a528fa8a7785d3c62dd014686f5f685379bf3561e8" exitCode=0
Oct 31 00:56:05.553453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.553344    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" event={"ID":"d34d3205-870b-476c-b376-964e6a79b049","Type":"ContainerDied","Data":"60992f2f477110423e9516a528fa8a7785d3c62dd014686f5f685379bf3561e8"}
Oct 31 00:56:05.553453 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.553398    4924 scope.go:122] "RemoveContainer" containerID="ff22f0e78bccc2397acbb8560977f497e1d61f4500ed5ec6730cab8fd1845dce"
Oct 31 00:56:05.656590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.656561    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:56:05.778652 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778608    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778670    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778705    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778780    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778846    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778878    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778912    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.778988    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779176 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.779062    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert\") pod \"d34d3205-870b-476c-b376-964e6a79b049\" (UID: \"d34d3205-870b-476c-b376-964e6a79b049\") "
Oct 31 00:56:05.779706 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.779667    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:05.779897 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.779875    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:05.780217 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.780152    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:05.780384 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.780362    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:05.780906 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.780874    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:05.782234 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.782203    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:05.782234 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.782313    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:56:05.782234 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.782620    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:05.786228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.786206    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4" pod "d34d3205-870b-476c-b376-964e6a79b049" (UID: "d34d3205-870b-476c-b376-964e6a79b049"). InnerVolumeSpecName "kube-api-access-xb8w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:05.881178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881140    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881167    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb8w4\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-kube-api-access-xb8w4\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881178 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881180    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881193    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881279    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d34d3205-870b-476c-b376-964e6a79b049-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881295    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d34d3205-870b-476c-b376-964e6a79b049-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881309    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881324    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d34d3205-870b-476c-b376-964e6a79b049-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:05.881407 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:05.881338    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d34d3205-870b-476c-b376-964e6a79b049-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:06.561714 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:06.561675    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46" event={"ID":"d34d3205-870b-476c-b376-964e6a79b049","Type":"ContainerDied","Data":"0c0e6876a3cace33bac374f08ae6309bcb69c9788455d7983b5e3e09028198cd"}
Oct 31 00:56:06.561931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:06.561760    4924 scope.go:122] "RemoveContainer" containerID="60992f2f477110423e9516a528fa8a7785d3c62dd014686f5f685379bf3561e8"
Oct 31 00:56:06.561931 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:06.561688    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"
Oct 31 00:56:06.584038 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:06.584013    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"]
Oct 31 00:56:06.590243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:06.590220    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-fb32d0-openshift-default-774dd6899c5wd46"]
Oct 31 00:56:07.765286 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:07.765253    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d34d3205-870b-476c-b376-964e6a79b049" path="/var/lib/kubelet/pods/d34d3205-870b-476c-b376-964e6a79b049/volumes"
Oct 31 00:56:13.002855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:13.002811    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:56:18.588323 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.588258    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:56:18.663043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.663007    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9\" (UniqueName: \"kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9\") pod \"269cc154-2460-4b1f-b4ac-80478f6fad8c\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") "
Oct 31 00:56:18.663189 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.663168    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume\") pod \"269cc154-2460-4b1f-b4ac-80478f6fad8c\" (UID: \"269cc154-2460-4b1f-b4ac-80478f6fad8c\") "
Oct 31 00:56:18.663572 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.663549    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume" pod "269cc154-2460-4b1f-b4ac-80478f6fad8c" (UID: "269cc154-2460-4b1f-b4ac-80478f6fad8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:18.665139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.665115    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9" pod "269cc154-2460-4b1f-b4ac-80478f6fad8c" (UID: "269cc154-2460-4b1f-b4ac-80478f6fad8c"). InnerVolumeSpecName "kube-api-access-2szs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:18.669318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.669296    4924 generic.go:356] "Generic (PLEG): container finished" podID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerID="7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632" exitCode=0
Oct 31 00:56:18.669418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.669357    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"
Oct 31 00:56:18.669466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.669360    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" event={"ID":"269cc154-2460-4b1f-b4ac-80478f6fad8c","Type":"ContainerDied","Data":"7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632"}
Oct 31 00:56:18.669509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.669476    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk" event={"ID":"269cc154-2460-4b1f-b4ac-80478f6fad8c","Type":"ContainerDied","Data":"101c783d36d8b2a4088800dd3b1d850b4ef94b1062de0a25ecd947a3eb203b46"}
Oct 31 00:56:18.669509 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.669498    4924 scope.go:122] "RemoveContainer" containerID="7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632"
Oct 31 00:56:18.680898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.680880    4924 scope.go:122] "RemoveContainer" containerID="7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632"
Oct 31 00:56:18.681173 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:56:18.681154    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632\": container with ID starting with 7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632 not found: ID does not exist" containerID="7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632"
Oct 31 00:56:18.681240 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.681192    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632"} err="failed to get container status \"7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632\": rpc error: code = NotFound desc = could not find container \"7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632\": container with ID starting with 7d450609f2ee352cce1ea236cb14de970cc54926308e689fe6c8e8dfd0389632 not found: ID does not exist"
Oct 31 00:56:18.692787 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.692761    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"]
Oct 31 00:56:18.696511 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.696487    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-fb32d0-7dd4fc88fwdcqk"]
Oct 31 00:56:18.764224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.764181    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269cc154-2460-4b1f-b4ac-80478f6fad8c-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:18.764224 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:18.764216    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2szs9\" (UniqueName: \"kubernetes.io/projected/269cc154-2460-4b1f-b4ac-80478f6fad8c-kube-api-access-2szs9\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:19.771656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:19.771622    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" path="/var/lib/kubelet/pods/269cc154-2460-4b1f-b4ac-80478f6fad8c/volumes"
Oct 31 00:56:20.380214 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.380174    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"]
Oct 31 00:56:20.380416 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.380381    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq" podUID="0671ec6a-6f7d-47e3-a253-2f7c53213996" containerName="manager" containerID="cri-o://10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d" gracePeriod=10
Oct 31 00:56:20.657267 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.657242    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:56:20.685689 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.685640    4924 generic.go:356] "Generic (PLEG): container finished" podID="0671ec6a-6f7d-47e3-a253-2f7c53213996" containerID="10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d" exitCode=0
Oct 31 00:56:20.685909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.685857    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq" event={"ID":"0671ec6a-6f7d-47e3-a253-2f7c53213996","Type":"ContainerDied","Data":"10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d"}
Oct 31 00:56:20.685909 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.685900    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq" event={"ID":"0671ec6a-6f7d-47e3-a253-2f7c53213996","Type":"ContainerDied","Data":"18e6979e85cc154401292b5c011118d42e5db7677af43c3c9cc74e5e01dc4385"}
Oct 31 00:56:20.686043 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.685926    4924 scope.go:122] "RemoveContainer" containerID="10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d"
Oct 31 00:56:20.686100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.686050    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"
Oct 31 00:56:20.700636 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.700608    4924 scope.go:122] "RemoveContainer" containerID="10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d"
Oct 31 00:56:20.700993 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:56:20.700973    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d\": container with ID starting with 10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d not found: ID does not exist" containerID="10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d"
Oct 31 00:56:20.701068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.701003    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d"} err="failed to get container status \"10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d\": rpc error: code = NotFound desc = could not find container \"10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d\": container with ID starting with 10215603b87d93c188de349579ff2b29aec255958f01aab9d6bd3fe0d616a75d not found: ID does not exist"
Oct 31 00:56:20.783550 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.783521    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt\" (UniqueName: \"kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt\") pod \"0671ec6a-6f7d-47e3-a253-2f7c53213996\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") "
Oct 31 00:56:20.783985 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.783581    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume\") pod \"0671ec6a-6f7d-47e3-a253-2f7c53213996\" (UID: \"0671ec6a-6f7d-47e3-a253-2f7c53213996\") "
Oct 31 00:56:20.784047 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.784021    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume" pod "0671ec6a-6f7d-47e3-a253-2f7c53213996" (UID: "0671ec6a-6f7d-47e3-a253-2f7c53213996"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:20.785613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.785591    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt" pod "0671ec6a-6f7d-47e3-a253-2f7c53213996" (UID: "0671ec6a-6f7d-47e3-a253-2f7c53213996"). InnerVolumeSpecName "kube-api-access-n5mnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:20.885448 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.885374    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5mnt\" (UniqueName: \"kubernetes.io/projected/0671ec6a-6f7d-47e3-a253-2f7c53213996-kube-api-access-n5mnt\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:20.885448 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:20.885403    4924 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0671ec6a-6f7d-47e3-a253-2f7c53213996-extensions-socket-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:21.010104 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:21.010067    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"]
Oct 31 00:56:21.013105 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:21.013075    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-tgctq"]
Oct 31 00:56:21.764883 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:56:21.764842    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0671ec6a-6f7d-47e3-a253-2f7c53213996" path="/var/lib/kubelet/pods/0671ec6a-6f7d-47e3-a253-2f7c53213996/volumes"
Oct 31 00:56:27.852046 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:56:27.851954    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:56:57.851920 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:56:57.851880    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:57:21.431212 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.431172    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"]
Oct 31 00:57:21.432484 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.432451    4924 cpu_manager.go:414] "containerMap: removing container" podUID="0671ec6a-6f7d-47e3-a253-2f7c53213996" containerName="manager"
Oct 31 00:57:21.432674 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.432659    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0671ec6a-6f7d-47e3-a253-2f7c53213996" containerName="manager"
Oct 31 00:57:21.432830 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.432816    4924 cpu_manager.go:414] "containerMap: removing container" podUID="4aadf8a4-a548-4bda-98ce-674397a4c95e" containerName="revoke-keys"
Oct 31 00:57:21.432930 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.432919    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aadf8a4-a548-4bda-98ce-674397a4c95e" containerName="revoke-keys"
Oct 31 00:57:21.433025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433013    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy"
Oct 31 00:57:21.433100 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433090    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy"
Oct 31 00:57:21.433172 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433163    4924 cpu_manager.go:414] "containerMap: removing container" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy"
Oct 31 00:57:21.433243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433233    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy"
Oct 31 00:57:21.433339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433330    4924 cpu_manager.go:414] "containerMap: removing container" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerName="payload-processing"
Oct 31 00:57:21.433411 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433401    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerName="payload-processing"
Oct 31 00:57:21.433784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433768    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aadf8a4-a548-4bda-98ce-674397a4c95e" containerName="revoke-keys"
Oct 31 00:57:21.433907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433896    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy"
Oct 31 00:57:21.434000 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.433991    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="d34d3205-870b-476c-b376-964e6a79b049" containerName="istio-proxy"
Oct 31 00:57:21.434074 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.434065    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="269cc154-2460-4b1f-b4ac-80478f6fad8c" containerName="payload-processing"
Oct 31 00:57:21.434160 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.434150    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="0671ec6a-6f7d-47e3-a253-2f7c53213996" containerName="manager"
Oct 31 00:57:21.438544 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.438524    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.446229 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.446201    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cbhxv\""
Oct 31 00:57:21.449369 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.449343    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"]
Oct 31 00:57:21.573404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.573367    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d12734-f949-4114-b639-543ab45e1f2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-hwnxg\" (UID: \"d7d12734-f949-4114-b639-543ab45e1f2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.573569 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.573421    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6zj\" (UniqueName: \"kubernetes.io/projected/d7d12734-f949-4114-b639-543ab45e1f2d-kube-api-access-cn6zj\") pod \"kuadrant-operator-controller-manager-678d87766f-hwnxg\" (UID: \"d7d12734-f949-4114-b639-543ab45e1f2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.674815 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.674774    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d12734-f949-4114-b639-543ab45e1f2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-hwnxg\" (UID: \"d7d12734-f949-4114-b639-543ab45e1f2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.674995 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.674834    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6zj\" (UniqueName: \"kubernetes.io/projected/d7d12734-f949-4114-b639-543ab45e1f2d-kube-api-access-cn6zj\") pod \"kuadrant-operator-controller-manager-678d87766f-hwnxg\" (UID: \"d7d12734-f949-4114-b639-543ab45e1f2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.675140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.675119    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d12734-f949-4114-b639-543ab45e1f2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-hwnxg\" (UID: \"d7d12734-f949-4114-b639-543ab45e1f2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.684530 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.684465    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6zj\" (UniqueName: \"kubernetes.io/projected/d7d12734-f949-4114-b639-543ab45e1f2d-kube-api-access-cn6zj\") pod \"kuadrant-operator-controller-manager-678d87766f-hwnxg\" (UID: \"d7d12734-f949-4114-b639-543ab45e1f2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:21.754438 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:21.754405    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:22.036451 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:22.036424    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"]
Oct 31 00:57:22.038656 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:57:22.038627    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d12734_f949_4114_b639_543ab45e1f2d.slice/crio-d2f551ac62f0fd7cfea79fce628f91ca5df43afd11f78ceab715c9ba7e72f9b1 WatchSource:0}: Error finding container d2f551ac62f0fd7cfea79fce628f91ca5df43afd11f78ceab715c9ba7e72f9b1: Status 404 returned error can't find the container with id d2f551ac62f0fd7cfea79fce628f91ca5df43afd11f78ceab715c9ba7e72f9b1
Oct 31 00:57:23.032605 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:23.032570    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg" event={"ID":"d7d12734-f949-4114-b639-543ab45e1f2d","Type":"ContainerStarted","Data":"880201c236c32ea765dd774a8a92d32db4752bf84099e266e040bfc8cc7f44bc"}
Oct 31 00:57:23.033048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:23.032629    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg" event={"ID":"d7d12734-f949-4114-b639-543ab45e1f2d","Type":"ContainerStarted","Data":"d2f551ac62f0fd7cfea79fce628f91ca5df43afd11f78ceab715c9ba7e72f9b1"}
Oct 31 00:57:23.033048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:23.032667    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:23.054405 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:23.054358    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg" podStartSLOduration=2.054344471 podStartE2EDuration="2.054344471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:23.052540776 +0000 UTC m=+1443.423324572" watchObservedRunningTime="2026-10-31 00:57:23.054344471 +0000 UTC m=+1443.425128259"
Oct 31 00:57:27.854311 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:57:27.854275    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:57:34.040919 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:34.040881    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hwnxg"
Oct 31 00:57:45.155008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.154967    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"]
Oct 31 00:57:45.158758 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.158724    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.165209 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.165184    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:57:45.165351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.165230    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:57:45.168090 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.168064    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"]
Oct 31 00:57:45.267688 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.267651    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.267900 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.267706    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.267900 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.267837    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppph4\" (UniqueName: \"kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.369318 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.369283    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.369523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.369328    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.369523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.369456    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppph4\" (UniqueName: \"kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.369977 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.369955    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.371653 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.371632    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.376947 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.376924    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppph4\" (UniqueName: \"kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4\") pod \"maas-api-revoke-sub-d576e68b-4zbp4\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.474908 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.474826    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:45.749975 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:45.749948    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"]
Oct 31 00:57:45.751779 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:57:45.751728    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae6ec8f_eade_485b_be9c_847dab16bb77.slice/crio-73e38b6ddf27de009453ae897aebb420135234a905f67f00a5421622f44652ab WatchSource:0}: Error finding container 73e38b6ddf27de009453ae897aebb420135234a905f67f00a5421622f44652ab: Status 404 returned error can't find the container with id 73e38b6ddf27de009453ae897aebb420135234a905f67f00a5421622f44652ab
Oct 31 00:57:46.217048 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:46.217010    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4" event={"ID":"fae6ec8f-eade-485b-be9c-847dab16bb77","Type":"ContainerStarted","Data":"65edf4e3f28b87eb068534583c4f0497f8753c947d3bfc69d00951bfd83c461c"}
Oct 31 00:57:46.217452 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:46.217065    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4" event={"ID":"fae6ec8f-eade-485b-be9c-847dab16bb77","Type":"ContainerStarted","Data":"73e38b6ddf27de009453ae897aebb420135234a905f67f00a5421622f44652ab"}
Oct 31 00:57:46.234026 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:46.233981    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4" podStartSLOduration=1.2339689759999999 podStartE2EDuration="1.233968976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:46.232132794 +0000 UTC m=+1466.602916582" watchObservedRunningTime="2026-10-31 00:57:46.233968976 +0000 UTC m=+1466.604752762"
Oct 31 00:57:47.225058 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:47.225024    4924 generic.go:356] "Generic (PLEG): container finished" podID="fae6ec8f-eade-485b-be9c-847dab16bb77" containerID="65edf4e3f28b87eb068534583c4f0497f8753c947d3bfc69d00951bfd83c461c" exitCode=0
Oct 31 00:57:47.225457 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:47.225107    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4" event={"ID":"fae6ec8f-eade-485b-be9c-847dab16bb77","Type":"ContainerDied","Data":"65edf4e3f28b87eb068534583c4f0497f8753c947d3bfc69d00951bfd83c461c"}
Oct 31 00:57:48.368287 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.368260    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:48.500535 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.500447    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca\") pod \"fae6ec8f-eade-485b-be9c-847dab16bb77\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") "
Oct 31 00:57:48.500696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.500601    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4\" (UniqueName: \"kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4\") pod \"fae6ec8f-eade-485b-be9c-847dab16bb77\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") "
Oct 31 00:57:48.500696 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.500664    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca\") pod \"fae6ec8f-eade-485b-be9c-847dab16bb77\" (UID: \"fae6ec8f-eade-485b-be9c-847dab16bb77\") "
Oct 31 00:57:48.500855 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.500831    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca" pod "fae6ec8f-eade-485b-be9c-847dab16bb77" (UID: "fae6ec8f-eade-485b-be9c-847dab16bb77"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:48.501016 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.500998    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:48.502692 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.502664    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca" pod "fae6ec8f-eade-485b-be9c-847dab16bb77" (UID: "fae6ec8f-eade-485b-be9c-847dab16bb77"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:48.502825 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.502718    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4" pod "fae6ec8f-eade-485b-be9c-847dab16bb77" (UID: "fae6ec8f-eade-485b-be9c-847dab16bb77"). InnerVolumeSpecName "kube-api-access-ppph4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:48.602231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.602187    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppph4\" (UniqueName: \"kubernetes.io/projected/fae6ec8f-eade-485b-be9c-847dab16bb77-kube-api-access-ppph4\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:48.602231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:48.602229    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fae6ec8f-eade-485b-be9c-847dab16bb77-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:49.237810 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:49.237780    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"
Oct 31 00:57:49.237989 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:49.237809    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4" event={"ID":"fae6ec8f-eade-485b-be9c-847dab16bb77","Type":"ContainerDied","Data":"73e38b6ddf27de009453ae897aebb420135234a905f67f00a5421622f44652ab"}
Oct 31 00:57:49.237989 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:57:49.237858    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73e38b6ddf27de009453ae897aebb420135234a905f67f00a5421622f44652ab"
Oct 31 00:57:57.853281 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:57:57.853229    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:58:04.065417 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.065333    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"]
Oct 31 00:58:04.065853 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.065812    4924 cpu_manager.go:414] "containerMap: removing container" podUID="fae6ec8f-eade-485b-be9c-847dab16bb77" containerName="revoke-keys"
Oct 31 00:58:04.065853 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.065828    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fae6ec8f-eade-485b-be9c-847dab16bb77" containerName="revoke-keys"
Oct 31 00:58:04.065978 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.065966    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="fae6ec8f-eade-485b-be9c-847dab16bb77" containerName="revoke-keys"
Oct 31 00:58:04.069526 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.069506    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.075590 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.075560    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 00:58:04.075841 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.075817    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 00:58:04.077487 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.077462    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"]
Oct 31 00:58:04.129454 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.129400    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.129649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.129566    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8hgt\" (UniqueName: \"kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.129649 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.129595    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.231140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.231099    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8hgt\" (UniqueName: \"kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.231140 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.231142    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.231351 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.231197    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.232025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.232004    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.233778 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.233760    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.239435 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.239407    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8hgt\" (UniqueName: \"kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt\") pod \"maas-api-revoke-sub-d2bc93bc-r4z8q\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.389685 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.389597    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:04.668029 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:04.667992    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"]
Oct 31 00:58:04.669810 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:58:04.669776    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3187b5_cd4b_4ec6_86b3_878648f0d311.slice/crio-570eb172c00a1d5c277b9d7275376271f2edd5e052ce5815723e09b5c7865d43 WatchSource:0}: Error finding container 570eb172c00a1d5c277b9d7275376271f2edd5e052ce5815723e09b5c7865d43: Status 404 returned error can't find the container with id 570eb172c00a1d5c277b9d7275376271f2edd5e052ce5815723e09b5c7865d43
Oct 31 00:58:05.395025 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:05.394986    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q" event={"ID":"df3187b5-cd4b-4ec6-86b3-878648f0d311","Type":"ContainerStarted","Data":"3ac0fbea4ff031c47434a743e13f71ac2e448bb3372ae59518f60181448e337e"}
Oct 31 00:58:05.395443 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:05.395049    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q" event={"ID":"df3187b5-cd4b-4ec6-86b3-878648f0d311","Type":"ContainerStarted","Data":"570eb172c00a1d5c277b9d7275376271f2edd5e052ce5815723e09b5c7865d43"}
Oct 31 00:58:05.411915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:05.411861    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q" podStartSLOduration=1.411846416 podStartE2EDuration="1.411846416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:05.410016277 +0000 UTC m=+1485.780800065" watchObservedRunningTime="2026-10-31 00:58:05.411846416 +0000 UTC m=+1485.782630247"
Oct 31 00:58:06.402617 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:06.402579    4924 generic.go:356] "Generic (PLEG): container finished" podID="df3187b5-cd4b-4ec6-86b3-878648f0d311" containerID="3ac0fbea4ff031c47434a743e13f71ac2e448bb3372ae59518f60181448e337e" exitCode=0
Oct 31 00:58:06.403031 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:06.402636    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q" event={"ID":"df3187b5-cd4b-4ec6-86b3-878648f0d311","Type":"ContainerDied","Data":"3ac0fbea4ff031c47434a743e13f71ac2e448bb3372ae59518f60181448e337e"}
Oct 31 00:58:07.557898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.557870    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:07.663515 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.663477    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt\" (UniqueName: \"kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt\") pod \"df3187b5-cd4b-4ec6-86b3-878648f0d311\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") "
Oct 31 00:58:07.663698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.663540    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca\") pod \"df3187b5-cd4b-4ec6-86b3-878648f0d311\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") "
Oct 31 00:58:07.663698 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.663652    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca\") pod \"df3187b5-cd4b-4ec6-86b3-878648f0d311\" (UID: \"df3187b5-cd4b-4ec6-86b3-878648f0d311\") "
Oct 31 00:58:07.664010 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.663983    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca" pod "df3187b5-cd4b-4ec6-86b3-878648f0d311" (UID: "df3187b5-cd4b-4ec6-86b3-878648f0d311"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:07.665802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.665776    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt" pod "df3187b5-cd4b-4ec6-86b3-878648f0d311" (UID: "df3187b5-cd4b-4ec6-86b3-878648f0d311"). InnerVolumeSpecName "kube-api-access-v8hgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:07.665899 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.665777    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca" pod "df3187b5-cd4b-4ec6-86b3-878648f0d311" (UID: "df3187b5-cd4b-4ec6-86b3-878648f0d311"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:58:07.764476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.764441    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:07.764476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.764469    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8hgt\" (UniqueName: \"kubernetes.io/projected/df3187b5-cd4b-4ec6-86b3-878648f0d311-kube-api-access-v8hgt\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:07.764476 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:07.764480    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df3187b5-cd4b-4ec6-86b3-878648f0d311-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:08.423531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:08.423496    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"
Oct 31 00:58:08.423714 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:08.423529    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q" event={"ID":"df3187b5-cd4b-4ec6-86b3-878648f0d311","Type":"ContainerDied","Data":"570eb172c00a1d5c277b9d7275376271f2edd5e052ce5815723e09b5c7865d43"}
Oct 31 00:58:08.423714 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:08.423580    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="570eb172c00a1d5c277b9d7275376271f2edd5e052ce5815723e09b5c7865d43"
Oct 31 00:58:16.066877 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.066840    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"]
Oct 31 00:58:16.067520 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.067502    4924 cpu_manager.go:414] "containerMap: removing container" podUID="df3187b5-cd4b-4ec6-86b3-878648f0d311" containerName="revoke-keys"
Oct 31 00:58:16.067564 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.067523    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df3187b5-cd4b-4ec6-86b3-878648f0d311" containerName="revoke-keys"
Oct 31 00:58:16.067673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.067661    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="df3187b5-cd4b-4ec6-86b3-878648f0d311" containerName="revoke-keys"
Oct 31 00:58:16.071228 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.071207    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.079654 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.079614    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-1919c8-openshift-default-dockercfg-dj4qm\""
Oct 31 00:58:16.082014 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.081987    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"]
Oct 31 00:58:16.152385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152343    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152385 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152387    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152406    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152452    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152570    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152618 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152608    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152679    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.152842 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152792    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.153023 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.152900    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt94l\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253470 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253431    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt94l\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253485    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253511    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253540    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253572    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253650    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.253700 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253698    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.254078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253736    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.254078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.253787    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.254078 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.254036    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.254296 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.254270    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.254465 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.254432    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.254593 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.254517    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.255017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.254995    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.256508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.256486    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.256876 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.256850    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.262929 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.262899    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.263068 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.263046    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt94l\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l\") pod \"e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") " pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.392299 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.392220    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:16.679208 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.679179    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"]
Oct 31 00:58:16.680957 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:58:16.680930    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f1c4cb_7d3c_41bc_8910_f5fed75f76e4.slice/crio-89bcbdf754dc9909cdf8b8141e5871ea0d58d3e59c76805294ea10a118dd42c7 WatchSource:0}: Error finding container 89bcbdf754dc9909cdf8b8141e5871ea0d58d3e59c76805294ea10a118dd42c7: Status 404 returned error can't find the container with id 89bcbdf754dc9909cdf8b8141e5871ea0d58d3e59c76805294ea10a118dd42c7
Oct 31 00:58:16.683851 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.683799    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:58:16.683953 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.683901    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:58:16.683999 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:16.683949    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:58:17.479871 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:17.479831    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" event={"ID":"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4","Type":"ContainerStarted","Data":"d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"}
Oct 31 00:58:17.480281 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:17.479927    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" event={"ID":"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4","Type":"ContainerStarted","Data":"89bcbdf754dc9909cdf8b8141e5871ea0d58d3e59c76805294ea10a118dd42c7"}
Oct 31 00:58:17.503080 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:17.503025    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podStartSLOduration=1.503011814 podStartE2EDuration="1.503011814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:17.49939516 +0000 UTC m=+1497.870178949" watchObservedRunningTime="2026-10-31 00:58:17.503011814 +0000 UTC m=+1497.873795674"
Oct 31 00:58:18.392402 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:18.392357    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:18.397582 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:18.397546    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:18.485523 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:18.485476    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:18.486729 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:18.486705    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:21.078301 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.078257    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"]
Oct 31 00:58:21.084616 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.084582    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.090976 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.090948    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"]
Oct 31 00:58:21.093225 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.093202    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-1919c8\""
Oct 31 00:58:21.142177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.142145    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"]
Oct 31 00:58:21.145864 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.145842    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.153531 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.153482    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-d72cf5d2-dockercfg-w8p8c\""
Oct 31 00:58:21.157030 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.157006    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"]
Oct 31 00:58:21.213915 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.213876    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvrm9\" (UniqueName: \"kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9\") pod \"payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.214185 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.214160    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ss5\" (UniqueName: \"kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5\") pod \"payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.214342 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.214326    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.214468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.214451    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.315466 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.315426    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrm9\" (UniqueName: \"kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9\") pod \"payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.315711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.315476    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ss5\" (UniqueName: \"kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5\") pod \"payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.315711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.315530    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.315711 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.315548    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.316387 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.316365    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.316430 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.316384    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.324870 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.324840    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrm9\" (UniqueName: \"kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9\") pod \"payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.324964 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.324885    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ss5\" (UniqueName: \"kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5\") pod \"payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.403895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.403817    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:21.459632 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.459463    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:21.686198 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.686170    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"]
Oct 31 00:58:21.687109 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:58:21.687068    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1dce39_7717_4d79_b61f_5e3ffd86b867.slice/crio-7f6f287a4a60b9e92b468faa887dd97edab25793dcf5e34737ff11e6469fcbee WatchSource:0}: Error finding container 7f6f287a4a60b9e92b468faa887dd97edab25793dcf5e34737ff11e6469fcbee: Status 404 returned error can't find the container with id 7f6f287a4a60b9e92b468faa887dd97edab25793dcf5e34737ff11e6469fcbee
Oct 31 00:58:21.752097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:21.752067    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"]
Oct 31 00:58:21.754610 ip-10-0-142-217 kubenswrapper[4924]: W1031 00:58:21.754564    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da0b1b4_65fe_4ed7_9bc1_776c83665d02.slice/crio-92510296fe32560e2afdb7c6c97e3f069cb265b3c64525b57fb8f054129976b4 WatchSource:0}: Error finding container 92510296fe32560e2afdb7c6c97e3f069cb265b3c64525b57fb8f054129976b4: Status 404 returned error can't find the container with id 92510296fe32560e2afdb7c6c97e3f069cb265b3c64525b57fb8f054129976b4
Oct 31 00:58:22.514802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.514757    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" event={"ID":"9e1dce39-7717-4d79-b61f-5e3ffd86b867","Type":"ContainerStarted","Data":"d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b"}
Oct 31 00:58:22.515289 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.514819    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" event={"ID":"9e1dce39-7717-4d79-b61f-5e3ffd86b867","Type":"ContainerStarted","Data":"7f6f287a4a60b9e92b468faa887dd97edab25793dcf5e34737ff11e6469fcbee"}
Oct 31 00:58:22.515289 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.514934    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:22.516468 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.516441    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" event={"ID":"5da0b1b4-65fe-4ed7-9bc1-776c83665d02","Type":"ContainerStarted","Data":"4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a"}
Oct 31 00:58:22.516621 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.516496    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:22.516621 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.516519    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" event={"ID":"5da0b1b4-65fe-4ed7-9bc1-776c83665d02","Type":"ContainerStarted","Data":"92510296fe32560e2afdb7c6c97e3f069cb265b3c64525b57fb8f054129976b4"}
Oct 31 00:58:22.535581 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.535532    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" podStartSLOduration=1.535519428 podStartE2EDuration="1.535519428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:22.533738611 +0000 UTC m=+1502.904522397" watchObservedRunningTime="2026-10-31 00:58:22.535519428 +0000 UTC m=+1502.906303216"
Oct 31 00:58:22.553447 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:22.553399    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" podStartSLOduration=1.553387101 podStartE2EDuration="1.553387101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:22.551908305 +0000 UTC m=+1502.922692096" watchObservedRunningTime="2026-10-31 00:58:22.553387101 +0000 UTC m=+1502.924170890"
Oct 31 00:58:26.461231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:26.461200    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:58:26.469508 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:26.469483    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:58:26.745914 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:26.745838    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 00:58:26.753721 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:26.753698    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 00:58:27.853764 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:58:27.853709    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:58:33.525691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:33.525651    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:58:33.526245 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:33.526081    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:58:33.803115 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:33.803085    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.155:15021/healthz/ready\": read tcp 10.132.0.2:36718->10.132.0.155:15021: read: connection reset by peer" start-of-body=
Oct 31 00:58:33.803298 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:33.803146    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.155:15021/healthz/ready\": read tcp 10.132.0.2:36718->10.132.0.155:15021: read: connection reset by peer"
Oct 31 00:58:34.599879 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:34.599842    4924 generic.go:356] "Generic (PLEG): container finished" podID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerID="d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd" exitCode=0
Oct 31 00:58:34.600316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:34.599914    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" event={"ID":"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4","Type":"ContainerDied","Data":"d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"}
Oct 31 00:58:34.600316 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:34.600304    4924 scope.go:122] "RemoveContainer" containerID="d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"
Oct 31 00:58:34.601579 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:34.601552    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:58:34.601641 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:34.601623    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:58:34.601689 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:34.601668    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 00:58:35.623097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:35.623064    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" event={"ID":"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4","Type":"ContainerStarted","Data":"de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4"}
Oct 31 00:58:36.392506 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:36.392468    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:36.405769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:36.392914    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused" start-of-body=
Oct 31 00:58:36.405769 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:36.393485    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused"
Oct 31 00:58:37.399784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:37.398171    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused" start-of-body=
Oct 31 00:58:37.399784 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:37.398237    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused"
Oct 31 00:58:38.395790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:38.393082    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused" start-of-body=
Oct 31 00:58:38.395790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:38.393160    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused"
Oct 31 00:58:39.397231 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:39.397191    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused" start-of-body=
Oct 31 00:58:39.397895 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:39.397862    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused"
Oct 31 00:58:40.398773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:40.397182    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused" start-of-body=
Oct 31 00:58:40.398773 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:40.397252    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.155:15021/healthz/ready\": dial tcp 10.132.0.155:15021: connect: connection refused"
Oct 31 00:58:41.397069 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:41.397023    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:41.397354 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:41.397331    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:41.398053 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:58:41.398033    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:58:57.853002 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:58:57.852961    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:59:27.867496 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:59:27.867451    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:59:29.293290 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:29.293254    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"]
Oct 31 00:59:29.324188 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:29.324143    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"]
Oct 31 00:59:34.294269 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:34.294140    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerName="payload-processing" containerID="cri-o://4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a" gracePeriod=25
Oct 31 00:59:34.325017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:34.324933    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerName="payload-pre-processing" containerID="cri-o://d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b" gracePeriod=25
Oct 31 00:59:37.625786 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:37.625741    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"]
Oct 31 00:59:37.626193 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:37.626001    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" containerID="cri-o://de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4" gracePeriod=30
Oct 31 00:59:41.400418 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:41.400376    4924 patch_prober.go:29] interesting pod/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:59:41.400898 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:41.400456    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:59:42.885881 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.885853    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:59:42.956948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.956854    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.956948 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.956936    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.956992    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957086    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957116    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957177 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957157    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957150    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:59:42.957397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957213    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957280    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957397 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957311    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket\") pod \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\" (UID: \"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4\") "
Oct 31 00:59:42.957591 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957444    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:59:42.957683 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957628    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:59:42.957741 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957659    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:42.957741 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957699    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:59:42.957741 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957722    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:42.957988 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.957966    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:59:42.959214 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.959189    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:59:42.959404 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.959377    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:42.959656 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.959634    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:59:42.959802 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:42.959783    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l" pod "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" (UID: "42f1c4cb-7d3c-41bc-8910-f5fed75f76e4"). InnerVolumeSpecName "kube-api-access-jt94l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:43.057420 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.057387    4924 generic.go:356] "Generic (PLEG): container finished" podID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerID="de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4" exitCode=0
Oct 31 00:59:43.057633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.057447    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" event={"ID":"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4","Type":"ContainerDied","Data":"de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4"}
Oct 31 00:59:43.057633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.057453    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"
Oct 31 00:59:43.057633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.057490    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27" event={"ID":"42f1c4cb-7d3c-41bc-8910-f5fed75f76e4","Type":"ContainerDied","Data":"89bcbdf754dc9909cdf8b8141e5871ea0d58d3e59c76805294ea10a118dd42c7"}
Oct 31 00:59:43.057633 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.057512    4924 scope.go:122] "RemoveContainer" containerID="de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4"
Oct 31 00:59:43.058327 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058303    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.058327 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058328    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.058489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058344    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.058489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058357    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.058489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058369    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.058489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058382    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt94l\" (UniqueName: \"kubernetes.io/projected/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-kube-api-access-jt94l\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.058489 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.058397    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:43.068926 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.068901    4924 scope.go:122] "RemoveContainer" containerID="d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"
Oct 31 00:59:43.077627 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.077604    4924 scope.go:122] "RemoveContainer" containerID="de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4"
Oct 31 00:59:43.077931 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:59:43.077908    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4\": container with ID starting with de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4 not found: ID does not exist" containerID="de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4"
Oct 31 00:59:43.077983 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.077939    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4"} err="failed to get container status \"de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4\": rpc error: code = NotFound desc = could not find container \"de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4\": container with ID starting with de4e9cb3cfab539c0db59843e16f475910c6aff6f5257aad0c05866df8f38fb4 not found: ID does not exist"
Oct 31 00:59:43.077983 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.077955    4924 scope.go:122] "RemoveContainer" containerID="d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"
Oct 31 00:59:43.078207 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:59:43.078186    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd\": container with ID starting with d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd not found: ID does not exist" containerID="d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"
Oct 31 00:59:43.078253 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.078218    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd"} err="failed to get container status \"d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd\": rpc error: code = NotFound desc = could not find container \"d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd\": container with ID starting with d13b0506b746df32247e719fac0c73c3647bca9b78772262ce3359130e5af9cd not found: ID does not exist"
Oct 31 00:59:43.085227 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.085205    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"]
Oct 31 00:59:43.087767 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.087731    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1919c8-openshift-default-6dcd6776bft2x27"]
Oct 31 00:59:43.524691 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.524651    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:59:43.524907 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.524691    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:59:43.769370 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:43.769330    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" path="/var/lib/kubelet/pods/42f1c4cb-7d3c-41bc-8910-f5fed75f76e4/volumes"
Oct 31 00:59:53.524339 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:53.524297    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:59:53.524857 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:53.524369    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:59:54.585571 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.585547    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:59:54.608139 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.608112    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:59:54.662896 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.662862    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5\" (UniqueName: \"kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5\") pod \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") "
Oct 31 00:59:54.663049 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.662995    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume\") pod \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\" (UID: \"5da0b1b4-65fe-4ed7-9bc1-776c83665d02\") "
Oct 31 00:59:54.663382 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.663359    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume" pod "5da0b1b4-65fe-4ed7-9bc1-776c83665d02" (UID: "5da0b1b4-65fe-4ed7-9bc1-776c83665d02"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:59:54.665017 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.664997    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5" pod "5da0b1b4-65fe-4ed7-9bc1-776c83665d02" (UID: "5da0b1b4-65fe-4ed7-9bc1-776c83665d02"). InnerVolumeSpecName "kube-api-access-c8ss5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:54.764675 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.764646    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume\") pod \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") "
Oct 31 00:59:54.764845 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.764762    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9\" (UniqueName: \"kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9\") pod \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\" (UID: \"9e1dce39-7717-4d79-b61f-5e3ffd86b867\") "
Oct 31 00:59:54.765079 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.765056    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume" pod "9e1dce39-7717-4d79-b61f-5e3ffd86b867" (UID: "9e1dce39-7717-4d79-b61f-5e3ffd86b867"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:59:54.765163 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.765147    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8ss5\" (UniqueName: \"kubernetes.io/projected/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-kube-api-access-c8ss5\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:54.765200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.765167    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e1dce39-7717-4d79-b61f-5e3ffd86b867-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:54.765200 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.765176    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da0b1b4-65fe-4ed7-9bc1-776c83665d02-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:54.766791 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.766771    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9" pod "9e1dce39-7717-4d79-b61f-5e3ffd86b867" (UID: "9e1dce39-7717-4d79-b61f-5e3ffd86b867"). InnerVolumeSpecName "kube-api-access-jvrm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:54.866621 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:54.866532    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvrm9\" (UniqueName: \"kubernetes.io/projected/9e1dce39-7717-4d79-b61f-5e3ffd86b867-kube-api-access-jvrm9\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:55.170943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.170859    4924 generic.go:356] "Generic (PLEG): container finished" podID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerID="4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a" exitCode=0
Oct 31 00:59:55.170943 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.170903    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"
Oct 31 00:59:55.171162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.170944    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" event={"ID":"5da0b1b4-65fe-4ed7-9bc1-776c83665d02","Type":"ContainerDied","Data":"4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a"}
Oct 31 00:59:55.171162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.170993    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc" event={"ID":"5da0b1b4-65fe-4ed7-9bc1-776c83665d02","Type":"ContainerDied","Data":"92510296fe32560e2afdb7c6c97e3f069cb265b3c64525b57fb8f054129976b4"}
Oct 31 00:59:55.171162 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.171018    4924 scope.go:122] "RemoveContainer" containerID="4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a"
Oct 31 00:59:55.172483 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.172460    4924 generic.go:356] "Generic (PLEG): container finished" podID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerID="d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b" exitCode=0
Oct 31 00:59:55.172613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.172485    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" event={"ID":"9e1dce39-7717-4d79-b61f-5e3ffd86b867","Type":"ContainerDied","Data":"d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b"}
Oct 31 00:59:55.172613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.172548    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t" event={"ID":"9e1dce39-7717-4d79-b61f-5e3ffd86b867","Type":"ContainerDied","Data":"7f6f287a4a60b9e92b468faa887dd97edab25793dcf5e34737ff11e6469fcbee"}
Oct 31 00:59:55.172613 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.172507    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"
Oct 31 00:59:55.184356 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.184335    4924 scope.go:122] "RemoveContainer" containerID="4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a"
Oct 31 00:59:55.184729 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:59:55.184704    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a\": container with ID starting with 4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a not found: ID does not exist" containerID="4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a"
Oct 31 00:59:55.184911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.184881    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a"} err="failed to get container status \"4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a\": rpc error: code = NotFound desc = could not find container \"4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a\": container with ID starting with 4182d0fb106d71cb6c3a6830039ae880fe53a38a12862950f304a3ed29fa0e6a not found: ID does not exist"
Oct 31 00:59:55.184911 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.184910    4924 scope.go:122] "RemoveContainer" containerID="d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b"
Oct 31 00:59:55.194097 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.194081    4924 scope.go:122] "RemoveContainer" containerID="d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b"
Oct 31 00:59:55.194340 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:59:55.194322    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b\": container with ID starting with d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b not found: ID does not exist" containerID="d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b"
Oct 31 00:59:55.194390 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.194345    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b"} err="failed to get container status \"d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b\": rpc error: code = NotFound desc = could not find container \"d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b\": container with ID starting with d7a717b6eedad752c12dfe3e917440cfb5d7f93df9cc968ec334a2811b2d831b not found: ID does not exist"
Oct 31 00:59:55.201694 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.201674    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"]
Oct 31 00:59:55.209373 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.209343    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-1919c8-59df94ch6t"]
Oct 31 00:59:55.214761 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.214730    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"]
Oct 31 00:59:55.217406 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.217382    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-1919c8-74d649f94rjlcc"]
Oct 31 00:59:55.763094 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.763060    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" path="/var/lib/kubelet/pods/5da0b1b4-65fe-4ed7-9bc1-776c83665d02/volumes"
Oct 31 00:59:55.763482 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:55.763450    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" path="/var/lib/kubelet/pods/9e1dce39-7717-4d79-b61f-5e3ffd86b867/volumes"
Oct 31 00:59:57.851580 ip-10-0-142-217 kubenswrapper[4924]: E1031 00:59:57.851545    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 00:59:59.483987 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.483940    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"]
Oct 31 00:59:59.484783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484729    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerName="payload-pre-processing"
Oct 31 00:59:59.484783 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484777    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerName="payload-pre-processing"
Oct 31 00:59:59.484957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484812    4924 cpu_manager.go:414] "containerMap: removing container" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerName="payload-processing"
Oct 31 00:59:59.484957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484822    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerName="payload-processing"
Oct 31 00:59:59.484957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484860    4924 cpu_manager.go:414] "containerMap: removing container" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy"
Oct 31 00:59:59.484957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484870    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy"
Oct 31 00:59:59.484957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484881    4924 cpu_manager.go:414] "containerMap: removing container" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy"
Oct 31 00:59:59.484957 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.484892    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy"
Oct 31 00:59:59.485831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.485035    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e1dce39-7717-4d79-b61f-5e3ffd86b867" containerName="payload-pre-processing"
Oct 31 00:59:59.485831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.485055    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="5da0b1b4-65fe-4ed7-9bc1-776c83665d02" containerName="payload-processing"
Oct 31 00:59:59.485831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.485070    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy"
Oct 31 00:59:59.485831 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.485080    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="42f1c4cb-7d3c-41bc-8910-f5fed75f76e4" containerName="istio-proxy"
Oct 31 00:59:59.488790 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.488759    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.494650 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.494615    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-2f32c431-kserve-self-signed-certs\""
Oct 31 00:59:59.500726 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.500695    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"]
Oct 31 00:59:59.612118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.612077    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.612118 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.612117    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75zc7\" (UniqueName: \"kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.612360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.612154    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.612360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.612301    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.612360 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.612331    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.612490 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.612439    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.713976    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714011 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714013    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714047    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714078    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714094    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75zc7\" (UniqueName: \"kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714283 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714114    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714484    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714568 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714536    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.714639 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.714568    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.716516 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.716486    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.716673 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.716655    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.722008 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.721988    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75zc7\" (UniqueName: \"kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7\") pod \"e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") " pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 00:59:59.804243 ip-10-0-142-217 kubenswrapper[4924]: I1031 00:59:59.804205    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:00.085339 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.085168    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"]
Oct 31 01:00:00.088327 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:00:00.088292    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c5035ff_1e13_4d18_abd2_15d4b5925f35.slice/crio-d07e7145da744b4f4c4d33785b9a7c613a04d69b97740fad111123d0cab4322e WatchSource:0}: Error finding container d07e7145da744b4f4c4d33785b9a7c613a04d69b97740fad111123d0cab4322e: Status 404 returned error can't find the container with id d07e7145da744b4f4c4d33785b9a7c613a04d69b97740fad111123d0cab4322e
Oct 31 01:00:00.135725 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.135699    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"]
Oct 31 01:00:00.138263 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.138238    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.146665 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.146632    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 01:00:00.164959 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.164922    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"]
Oct 31 01:00:00.205286 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.205243    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" event={"ID":"2c5035ff-1e13-4d18-abd2-15d4b5925f35","Type":"ContainerStarted","Data":"e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce"}
Oct 31 01:00:00.205286 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.205288    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" event={"ID":"2c5035ff-1e13-4d18-abd2-15d4b5925f35","Type":"ContainerStarted","Data":"d07e7145da744b4f4c4d33785b9a7c613a04d69b97740fad111123d0cab4322e"}
Oct 31 01:00:00.220142 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.220107    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890140-6dqp4\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.220300 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.220281    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7pr7\" (UniqueName: \"kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7\") pod \"maas-api-key-cleanup-29890140-6dqp4\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.223650 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.223607    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" podStartSLOduration=1.223592465 podStartE2EDuration="1.223592465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:00.22255544 +0000 UTC m=+1600.593339220" watchObservedRunningTime="2026-10-31 01:00:00.223592465 +0000 UTC m=+1600.594376261"
Oct 31 01:00:00.321725 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.321685    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890140-6dqp4\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.321925 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.321907    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7pr7\" (UniqueName: \"kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7\") pod \"maas-api-key-cleanup-29890140-6dqp4\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.322511 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.322491    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890140-6dqp4\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.330362 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.330338    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7pr7\" (UniqueName: \"kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7\") pod \"maas-api-key-cleanup-29890140-6dqp4\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.466991 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.466914    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:00.743817 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.743790    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"]
Oct 31 01:00:00.746101 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:00:00.746049    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ff906a_13b8_4618_9173_2aa68d7dd5b3.slice/crio-c43e78674f9958cf44d518b83e0fb6bf2bd9f3d32e59175516ca8e4c349c4f81 WatchSource:0}: Error finding container c43e78674f9958cf44d518b83e0fb6bf2bd9f3d32e59175516ca8e4c349c4f81: Status 404 returned error can't find the container with id c43e78674f9958cf44d518b83e0fb6bf2bd9f3d32e59175516ca8e4c349c4f81
Oct 31 01:00:00.805166 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:00.805131    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:01.213989 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:01.213936    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerStarted","Data":"8d2c7e2fcec2d12eb94c66e62f6bad90a08bb33e5134f1aa8e1cf4d9fc2a8dee"}
Oct 31 01:00:01.214166 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:01.214005    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerStarted","Data":"c43e78674f9958cf44d518b83e0fb6bf2bd9f3d32e59175516ca8e4c349c4f81"}
Oct 31 01:00:01.230919 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:01.230863    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" podStartSLOduration=1.230848817 podStartE2EDuration="1.230848817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:01.228908347 +0000 UTC m=+1601.599692138" watchObservedRunningTime="2026-10-31 01:00:01.230848817 +0000 UTC m=+1601.601632607"
Oct 31 01:00:02.229590 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:02.229546    4924 generic.go:356] "Generic (PLEG): container finished" podID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerID="8d2c7e2fcec2d12eb94c66e62f6bad90a08bb33e5134f1aa8e1cf4d9fc2a8dee" exitCode=60
Oct 31 01:00:02.231868 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:02.231837    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerDied","Data":"8d2c7e2fcec2d12eb94c66e62f6bad90a08bb33e5134f1aa8e1cf4d9fc2a8dee"}
Oct 31 01:00:02.232097 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:02.231884    4924 scope.go:122] "RemoveContainer" containerID="8d2c7e2fcec2d12eb94c66e62f6bad90a08bb33e5134f1aa8e1cf4d9fc2a8dee"
Oct 31 01:00:03.242772 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:03.242499    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerStarted","Data":"a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888"}
Oct 31 01:00:04.269770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:04.269434    4924 generic.go:356] "Generic (PLEG): container finished" podID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerID="a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888" exitCode=60
Oct 31 01:00:04.269770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:04.269623    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerDied","Data":"a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888"}
Oct 31 01:00:04.269770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:04.269692    4924 scope.go:122] "RemoveContainer" containerID="8d2c7e2fcec2d12eb94c66e62f6bad90a08bb33e5134f1aa8e1cf4d9fc2a8dee"
Oct 31 01:00:04.270432 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:04.270204    4924 scope.go:122] "RemoveContainer" containerID="a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888"
Oct 31 01:00:04.273777 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:04.270529    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890140-6dqp4_odh-ai-gateway-infra(10ff906a-13b8-4618-9173-2aa68d7dd5b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3"
Oct 31 01:00:05.294777 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:05.294229    4924 scope.go:122] "RemoveContainer" containerID="a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888"
Oct 31 01:00:05.294777 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:05.294595    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890140-6dqp4_odh-ai-gateway-infra(10ff906a-13b8-4618-9173-2aa68d7dd5b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3"
Oct 31 01:00:09.804826 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:09.804782    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:09.817739 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:09.817711    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:10.334958 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:10.334919    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:17.877777 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:17.874016    4924 scope.go:122] "RemoveContainer" containerID="a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888"
Oct 31 01:00:18.961771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:18.960871    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:00:18.961771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:18.960946    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:00:19.489763 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:19.489697    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerStarted","Data":"d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75"}
Oct 31 01:00:19.507771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:19.490550    4924 scope.go:122] "RemoveContainer" containerID="d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75"
Oct 31 01:00:19.510447 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:19.510403    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890140-6dqp4_odh-ai-gateway-infra(10ff906a-13b8-4618-9173-2aa68d7dd5b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3"
Oct 31 01:00:19.804210 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:19.804169    4924 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ff906a_13b8_4618_9173_2aa68d7dd5b3.slice/crio-d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 01:00:20.498503 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:20.498461    4924 generic.go:356] "Generic (PLEG): container finished" podID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerID="d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75" exitCode=60
Oct 31 01:00:20.498956 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:20.498546    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerDied","Data":"d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75"}
Oct 31 01:00:20.498956 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:20.498612    4924 scope.go:122] "RemoveContainer" containerID="a2d817f847c1eb6960dfce01e50f4df85aeec54ce516b93a5ea8fd50a194f888"
Oct 31 01:00:20.499100 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:20.498985    4924 scope.go:122] "RemoveContainer" containerID="d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75"
Oct 31 01:00:20.499264 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:20.499237    4924 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890140-6dqp4_odh-ai-gateway-infra(10ff906a-13b8-4618-9173-2aa68d7dd5b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3"
Oct 31 01:00:20.540494 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:20.540450    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"]
Oct 31 01:00:21.651034 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.651000    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:21.749770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.749723    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca\") pod \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") "
Oct 31 01:00:21.749953 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.749920    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7\" (UniqueName: \"kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7\") pod \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\" (UID: \"10ff906a-13b8-4618-9173-2aa68d7dd5b3\") "
Oct 31 01:00:21.750081 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.750056    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca" pod "10ff906a-13b8-4618-9173-2aa68d7dd5b3" (UID: "10ff906a-13b8-4618-9173-2aa68d7dd5b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:00:21.750215 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.750198    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ff906a-13b8-4618-9173-2aa68d7dd5b3-maas-api-service-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:21.752040 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.752014    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7" pod "10ff906a-13b8-4618-9173-2aa68d7dd5b3" (UID: "10ff906a-13b8-4618-9173-2aa68d7dd5b3"). InnerVolumeSpecName "kube-api-access-v7pr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:21.851381 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:21.851336    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7pr7\" (UniqueName: \"kubernetes.io/projected/10ff906a-13b8-4618-9173-2aa68d7dd5b3-kube-api-access-v7pr7\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:22.514217 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:22.514187    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"
Oct 31 01:00:22.514403 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:22.514217    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4" event={"ID":"10ff906a-13b8-4618-9173-2aa68d7dd5b3","Type":"ContainerDied","Data":"c43e78674f9958cf44d518b83e0fb6bf2bd9f3d32e59175516ca8e4c349c4f81"}
Oct 31 01:00:22.514403 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:22.514276    4924 scope.go:122] "RemoveContainer" containerID="d46bcc1bccb360a0cae0f33c75c4deb55532d59e2cd357aada4419181152ec75"
Oct 31 01:00:22.531656 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:22.531624    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"]
Oct 31 01:00:22.535340 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:22.535313    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890140-6dqp4"]
Oct 31 01:00:23.763149 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:23.763114    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" path="/var/lib/kubelet/pods/10ff906a-13b8-4618-9173-2aa68d7dd5b3/volumes"
Oct 31 01:00:27.565548 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:27.565510    4924 scope.go:122] "RemoveContainer" containerID="109b180f7fbe5ab73cf6611f705455abb49373925d69cb775fe0bb198c067435"
Oct 31 01:00:27.855355 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:27.855259    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:00:28.097381 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:28.097348    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"]
Oct 31 01:00:29.773100 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773064    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"]
Oct 31 01:00:29.773728 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773703    4924 cpu_manager.go:414] "containerMap: removing container" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.773866 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773728    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.773866 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773764    4924 cpu_manager.go:414] "containerMap: removing container" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.773866 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773775    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.774050 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773935    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.774050 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773952    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.774050 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.773964    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:29.786856 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.786827    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"]
Oct 31 01:00:29.787150 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.787130    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:29.798113 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.797091    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:00:29.798113 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.798076    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 01:00:29.950982 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.950927    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:29.951200 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.951068    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:29.951200 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:29.951192    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgwkd\" (UniqueName: \"kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.053460 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.052354    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.053460 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.052439    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.053460 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.052501    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgwkd\" (UniqueName: \"kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.054699 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.054668    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.063187 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.063161    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgwkd\" (UniqueName: \"kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.085409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.085378    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-x9cqz\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:30.120774 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:30.120723    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:31.149769 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:31.149344    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"]
Oct 31 01:00:31.223484 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:00:31.222040    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613f15ee_fa77_4623_b3a6_1ede827bd225.slice/crio-e2a61435ef7870ca63da0923baee7d2159c686333281c17230de9a7e9bb30e1e WatchSource:0}: Error finding container e2a61435ef7870ca63da0923baee7d2159c686333281c17230de9a7e9bb30e1e: Status 404 returned error can't find the container with id e2a61435ef7870ca63da0923baee7d2159c686333281c17230de9a7e9bb30e1e
Oct 31 01:00:31.849826 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:31.849279    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz" event={"ID":"613f15ee-fa77-4623-b3a6-1ede827bd225","Type":"ContainerStarted","Data":"08ff18b5285ec1c93570e7f958c86ed5860d612bdd172d3598a7051cc9f4ccc2"}
Oct 31 01:00:31.849826 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:31.849350    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz" event={"ID":"613f15ee-fa77-4623-b3a6-1ede827bd225","Type":"ContainerStarted","Data":"e2a61435ef7870ca63da0923baee7d2159c686333281c17230de9a7e9bb30e1e"}
Oct 31 01:00:31.907771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:31.896835    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz" podStartSLOduration=2.896815576 podStartE2EDuration="2.896815576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:31.896241218 +0000 UTC m=+1632.267025009" watchObservedRunningTime="2026-10-31 01:00:31.896815576 +0000 UTC m=+1632.267599367"
Oct 31 01:00:32.861348 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:32.861310    4924 generic.go:356] "Generic (PLEG): container finished" podID="613f15ee-fa77-4623-b3a6-1ede827bd225" containerID="08ff18b5285ec1c93570e7f958c86ed5860d612bdd172d3598a7051cc9f4ccc2" exitCode=0
Oct 31 01:00:32.861847 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:32.861396    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz" event={"ID":"613f15ee-fa77-4623-b3a6-1ede827bd225","Type":"ContainerDied","Data":"08ff18b5285ec1c93570e7f958c86ed5860d612bdd172d3598a7051cc9f4ccc2"}
Oct 31 01:00:34.703109 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.703071    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:34.752176 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.752132    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca\") pod \"613f15ee-fa77-4623-b3a6-1ede827bd225\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") "
Oct 31 01:00:34.752562 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.752537    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca\") pod \"613f15ee-fa77-4623-b3a6-1ede827bd225\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") "
Oct 31 01:00:34.754326 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.754300    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd\" (UniqueName: \"kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd\") pod \"613f15ee-fa77-4623-b3a6-1ede827bd225\" (UID: \"613f15ee-fa77-4623-b3a6-1ede827bd225\") "
Oct 31 01:00:34.771770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.753425    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca" pod "613f15ee-fa77-4623-b3a6-1ede827bd225" (UID: "613f15ee-fa77-4623-b3a6-1ede827bd225"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:00:34.818296 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.818251    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca" pod "613f15ee-fa77-4623-b3a6-1ede827bd225" (UID: "613f15ee-fa77-4623-b3a6-1ede827bd225"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:00:34.819880 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.819825    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd" pod "613f15ee-fa77-4623-b3a6-1ede827bd225" (UID: "613f15ee-fa77-4623-b3a6-1ede827bd225"). InnerVolumeSpecName "kube-api-access-rgwkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:34.902831 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.899324    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:34.902831 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.899614    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/613f15ee-fa77-4623-b3a6-1ede827bd225-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:34.902831 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:34.899643    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgwkd\" (UniqueName: \"kubernetes.io/projected/613f15ee-fa77-4623-b3a6-1ede827bd225-kube-api-access-rgwkd\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:35.001417 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:35.001374    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz" event={"ID":"613f15ee-fa77-4623-b3a6-1ede827bd225","Type":"ContainerDied","Data":"e2a61435ef7870ca63da0923baee7d2159c686333281c17230de9a7e9bb30e1e"}
Oct 31 01:00:35.001757 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:35.001710    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2a61435ef7870ca63da0923baee7d2159c686333281c17230de9a7e9bb30e1e"
Oct 31 01:00:35.007126 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:35.007100    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"
Oct 31 01:00:36.072459 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:36.072420    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"]
Oct 31 01:00:36.088663 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:36.088623    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f9b0664a-s95q8"]
Oct 31 01:00:37.771051 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:37.771014    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c72824a-6c3c-49bc-998b-6d8830fb125a" path="/var/lib/kubelet/pods/1c72824a-6c3c-49bc-998b-6d8830fb125a/volumes"
Oct 31 01:00:43.130113 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.130053    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" podUID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" containerName="main" containerID="cri-o://e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce" gracePeriod=45
Oct 31 01:00:43.324770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.324460    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" podUID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" containerName="main" probeResult="failure" output="Get \"https://10.132.0.158:8000/ready\": dial tcp 10.132.0.158:8000: connect: connection refused"
Oct 31 01:00:43.486231 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.486204    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:43.659372 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659205    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm\") pod \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") "
Oct 31 01:00:43.659372 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659355    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs\") pod \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") "
Oct 31 01:00:43.659625 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659389    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home\") pod \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") "
Oct 31 01:00:43.659625 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659424    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache\") pod \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") "
Oct 31 01:00:43.659625 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659491    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir\") pod \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") "
Oct 31 01:00:43.659625 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659536    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7\" (UniqueName: \"kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7\") pod \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\" (UID: \"2c5035ff-1e13-4d18-abd2-15d4b5925f35\") "
Oct 31 01:00:43.659897 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.659855    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home" pod "2c5035ff-1e13-4d18-abd2-15d4b5925f35" (UID: "2c5035ff-1e13-4d18-abd2-15d4b5925f35"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:43.660562 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.660168    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir" pod "2c5035ff-1e13-4d18-abd2-15d4b5925f35" (UID: "2c5035ff-1e13-4d18-abd2-15d4b5925f35"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:43.660562 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.660537    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache" pod "2c5035ff-1e13-4d18-abd2-15d4b5925f35" (UID: "2c5035ff-1e13-4d18-abd2-15d4b5925f35"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:43.660979 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.660891    4924 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-home\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:43.660979 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.660916    4924 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-model-cache\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:43.660979 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.660955    4924 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tmp-dir\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:43.662579 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.662550    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm" pod "2c5035ff-1e13-4d18-abd2-15d4b5925f35" (UID: "2c5035ff-1e13-4d18-abd2-15d4b5925f35"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:43.663429 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.663388    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7" pod "2c5035ff-1e13-4d18-abd2-15d4b5925f35" (UID: "2c5035ff-1e13-4d18-abd2-15d4b5925f35"). InnerVolumeSpecName "kube-api-access-75zc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:43.664163 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.664141    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs" pod "2c5035ff-1e13-4d18-abd2-15d4b5925f35" (UID: "2c5035ff-1e13-4d18-abd2-15d4b5925f35"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:00:43.762233 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.762156    4924 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c5035ff-1e13-4d18-abd2-15d4b5925f35-tls-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:43.762233 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.762193    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75zc7\" (UniqueName: \"kubernetes.io/projected/2c5035ff-1e13-4d18-abd2-15d4b5925f35-kube-api-access-75zc7\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:43.762233 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:43.762207    4924 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c5035ff-1e13-4d18-abd2-15d4b5925f35-dshm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:44.127860 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.127813    4924 generic.go:356] "Generic (PLEG): container finished" podID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" containerID="e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce" exitCode=0
Oct 31 01:00:44.128049 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.127974    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" event={"ID":"2c5035ff-1e13-4d18-abd2-15d4b5925f35","Type":"ContainerDied","Data":"e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce"}
Oct 31 01:00:44.128049 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.128017    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t" event={"ID":"2c5035ff-1e13-4d18-abd2-15d4b5925f35","Type":"ContainerDied","Data":"d07e7145da744b4f4c4d33785b9a7c613a04d69b97740fad111123d0cab4322e"}
Oct 31 01:00:44.128049 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.128041    4924 scope.go:122] "RemoveContainer" containerID="e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce"
Oct 31 01:00:44.128262 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.128229    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"
Oct 31 01:00:44.143874 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.143836    4924 scope.go:122] "RemoveContainer" containerID="e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce"
Oct 31 01:00:44.144296 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:44.144260    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce\": container with ID starting with e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce not found: ID does not exist" containerID="e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce"
Oct 31 01:00:44.144380 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.144331    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce"} err="failed to get container status \"e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce\": rpc error: code = NotFound desc = could not find container \"e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce\": container with ID starting with e12752ef03e0e9b424b6ed467074c75f65dcb16ba12fa142559ce835ac36c6ce not found: ID does not exist"
Oct 31 01:00:44.161158 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.161111    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"]
Oct 31 01:00:44.167810 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:44.167764    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-2f32c431-kserve-686cb4f4f7-2z46t"]
Oct 31 01:00:45.777995 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:45.777956    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" path="/var/lib/kubelet/pods/2c5035ff-1e13-4d18-abd2-15d4b5925f35/volumes"
Oct 31 01:00:50.220772 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.220720    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"]
Oct 31 01:00:50.221363 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221342    4924 cpu_manager.go:414] "containerMap: removing container" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:50.221453 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221368    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ff906a-13b8-4618-9173-2aa68d7dd5b3" containerName="cleanup"
Oct 31 01:00:50.221453 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221427    4924 cpu_manager.go:414] "containerMap: removing container" podUID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" containerName="main"
Oct 31 01:00:50.221453 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221435    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" containerName="main"
Oct 31 01:00:50.221453 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221446    4924 cpu_manager.go:414] "containerMap: removing container" podUID="613f15ee-fa77-4623-b3a6-1ede827bd225" containerName="revoke-keys"
Oct 31 01:00:50.221671 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221454    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="613f15ee-fa77-4623-b3a6-1ede827bd225" containerName="revoke-keys"
Oct 31 01:00:50.221671 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221564    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c5035ff-1e13-4d18-abd2-15d4b5925f35" containerName="main"
Oct 31 01:00:50.221671 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.221578    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="613f15ee-fa77-4623-b3a6-1ede827bd225" containerName="revoke-keys"
Oct 31 01:00:50.226458 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.226435    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.230022 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.229998    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:00:50.230174 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.230119    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 01:00:50.233131 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.233098    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"]
Oct 31 01:00:50.367550 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.367513    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.367705 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.367555    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wjm\" (UniqueName: \"kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.367705 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.367688    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.468963 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.468924    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.469165 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.468971    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wjm\" (UniqueName: \"kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.469165 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.469062    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.469655 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.469628    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.471516 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.471460    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.476127 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.476107    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wjm\" (UniqueName: \"kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm\") pod \"maas-api-revoke-sub-28b214c4-gs4zq\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.542258 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.542221    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:50.818016 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:50.817990    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"]
Oct 31 01:00:50.819415 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:00:50.819390    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb199a7_62f3_4139_89ae_8d0f831c2085.slice/crio-720058e2415a06f52370ee5c356b2a172de8e9704d71a68ae2c641396743d562 WatchSource:0}: Error finding container 720058e2415a06f52370ee5c356b2a172de8e9704d71a68ae2c641396743d562: Status 404 returned error can't find the container with id 720058e2415a06f52370ee5c356b2a172de8e9704d71a68ae2c641396743d562
Oct 31 01:00:51.215065 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:51.215033    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq" event={"ID":"acb199a7-62f3-4139-89ae-8d0f831c2085","Type":"ContainerStarted","Data":"3502717232bc3b054ed74a81cdacbc016b250543fa10ed95a81d9cef728d0eb7"}
Oct 31 01:00:51.215241 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:51.215085    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq" event={"ID":"acb199a7-62f3-4139-89ae-8d0f831c2085","Type":"ContainerStarted","Data":"720058e2415a06f52370ee5c356b2a172de8e9704d71a68ae2c641396743d562"}
Oct 31 01:00:51.233589 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:51.233541    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq" podStartSLOduration=1.233529322 podStartE2EDuration="1.233529322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:51.232934091 +0000 UTC m=+1651.603717876" watchObservedRunningTime="2026-10-31 01:00:51.233529322 +0000 UTC m=+1651.604313140"
Oct 31 01:00:52.222381 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:52.222344    4924 generic.go:356] "Generic (PLEG): container finished" podID="acb199a7-62f3-4139-89ae-8d0f831c2085" containerID="3502717232bc3b054ed74a81cdacbc016b250543fa10ed95a81d9cef728d0eb7" exitCode=0
Oct 31 01:00:52.222564 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:52.222421    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq" event={"ID":"acb199a7-62f3-4139-89ae-8d0f831c2085","Type":"ContainerDied","Data":"3502717232bc3b054ed74a81cdacbc016b250543fa10ed95a81d9cef728d0eb7"}
Oct 31 01:00:53.366570 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.366538    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:53.498641 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.498556    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca\") pod \"acb199a7-62f3-4139-89ae-8d0f831c2085\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") "
Oct 31 01:00:53.498641 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.498598    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca\") pod \"acb199a7-62f3-4139-89ae-8d0f831c2085\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") "
Oct 31 01:00:53.498641 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.498638    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm\" (UniqueName: \"kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm\") pod \"acb199a7-62f3-4139-89ae-8d0f831c2085\" (UID: \"acb199a7-62f3-4139-89ae-8d0f831c2085\") "
Oct 31 01:00:53.499035 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.499009    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca" pod "acb199a7-62f3-4139-89ae-8d0f831c2085" (UID: "acb199a7-62f3-4139-89ae-8d0f831c2085"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:00:53.500801 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.500775    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca" pod "acb199a7-62f3-4139-89ae-8d0f831c2085" (UID: "acb199a7-62f3-4139-89ae-8d0f831c2085"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:00:53.500891 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.500838    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm" pod "acb199a7-62f3-4139-89ae-8d0f831c2085" (UID: "acb199a7-62f3-4139-89ae-8d0f831c2085"). InnerVolumeSpecName "kube-api-access-94wjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:53.599806 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.599774    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:53.599806 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.599805    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acb199a7-62f3-4139-89ae-8d0f831c2085-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:53.599990 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:53.599814    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94wjm\" (UniqueName: \"kubernetes.io/projected/acb199a7-62f3-4139-89ae-8d0f831c2085-kube-api-access-94wjm\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:54.234863 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:54.234783    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"
Oct 31 01:00:54.234863 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:54.234795    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq" event={"ID":"acb199a7-62f3-4139-89ae-8d0f831c2085","Type":"ContainerDied","Data":"720058e2415a06f52370ee5c356b2a172de8e9704d71a68ae2c641396743d562"}
Oct 31 01:00:54.234863 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:00:54.234840    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="720058e2415a06f52370ee5c356b2a172de8e9704d71a68ae2c641396743d562"
Oct 31 01:00:57.852517 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:00:57.852477    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:01:05.032643 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:05.032512    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"]
Oct 31 01:01:05.034950 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:05.034927    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-rb7gb"]
Oct 31 01:01:05.763661 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:05.763621    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4aadf8a4-a548-4bda-98ce-674397a4c95e" path="/var/lib/kubelet/pods/4aadf8a4-a548-4bda-98ce-674397a4c95e/volumes"
Oct 31 01:01:09.825240 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.825205    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"]
Oct 31 01:01:09.825874 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.825855    4924 cpu_manager.go:414] "containerMap: removing container" podUID="acb199a7-62f3-4139-89ae-8d0f831c2085" containerName="revoke-keys"
Oct 31 01:01:09.825983 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.825876    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acb199a7-62f3-4139-89ae-8d0f831c2085" containerName="revoke-keys"
Oct 31 01:01:09.826046 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.826035    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="acb199a7-62f3-4139-89ae-8d0f831c2085" containerName="revoke-keys"
Oct 31 01:01:09.830457 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.830437    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.850072 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.850049    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7e1e1fc6-openshift-default-dockercfg-5wx4r\""
Oct 31 01:01:09.863323 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863296    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863418 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863337    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863418 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863402    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863505 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863456    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863505 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863477    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8hbs\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863647 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863627    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863710 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863669    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863851 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863834    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.863927 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.863865    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.956550 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.956509    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"]
Oct 31 01:01:09.964862 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.964834    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.964862 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.964865    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.964901    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.964934    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.964968    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965013    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965031    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8hbs\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965074    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965102 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965096    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965489 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965326    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965637 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965610    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965830 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965726    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965830 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965816    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.965986 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.965947    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.968032 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.968001    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.968264 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.968245    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.992687 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.992651    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8hbs\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:09.992687 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:09.992676    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token\") pod \"e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") " pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:10.144786 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:10.144667    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:10.431316 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:10.431282    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"]
Oct 31 01:01:10.433230 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:01:10.433198    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445e3f60_f5f0_4e4b_b452_c2fd644f638a.slice/crio-e18a8788344ebbf44d3f727eb15542336ebf6e6f822c5a24f709b9e2f33d48c1 WatchSource:0}: Error finding container e18a8788344ebbf44d3f727eb15542336ebf6e6f822c5a24f709b9e2f33d48c1: Status 404 returned error can't find the container with id e18a8788344ebbf44d3f727eb15542336ebf6e6f822c5a24f709b9e2f33d48c1
Oct 31 01:01:10.435619 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:10.435585    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:10.435695 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:10.435655    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:10.435737 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:10.435706    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:11.335240 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:11.335201    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" event={"ID":"445e3f60-f5f0-4e4b-b452-c2fd644f638a","Type":"ContainerStarted","Data":"478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"}
Oct 31 01:01:11.335642 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:11.335261    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" event={"ID":"445e3f60-f5f0-4e4b-b452-c2fd644f638a","Type":"ContainerStarted","Data":"e18a8788344ebbf44d3f727eb15542336ebf6e6f822c5a24f709b9e2f33d48c1"}
Oct 31 01:01:11.360255 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:11.360204    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podStartSLOduration=2.360188839 podStartE2EDuration="2.360188839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:11.359076626 +0000 UTC m=+1671.729860415" watchObservedRunningTime="2026-10-31 01:01:11.360188839 +0000 UTC m=+1671.730972684"
Oct 31 01:01:12.145784 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:12.145733    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:12.150458 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:12.150428    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:12.343317 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:12.343285    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:12.344326 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:12.344301    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:14.284790 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.284741    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"]
Oct 31 01:01:14.288490 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.288472    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.295785 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.295348    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7e1e1fc6-dockercfg-9lcn4\""
Oct 31 01:01:14.295785 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.295401    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7e1e1fc6\""
Oct 31 01:01:14.299057 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.299034    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"]
Oct 31 01:01:14.323378 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.323346    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"]
Oct 31 01:01:14.327076 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.327054    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.336602 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.336576    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"]
Oct 31 01:01:14.411856 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.411812    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw4m5\" (UniqueName: \"kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5\") pod \"payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") " pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.412060 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.411886    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume\") pod \"payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") " pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.412158 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.412124    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r85wx\" (UniqueName: \"kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx\") pod \"payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.412415 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.412394    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.513572 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.513531    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r85wx\" (UniqueName: \"kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx\") pod \"payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.513849 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.513590    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.513849 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.513661    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw4m5\" (UniqueName: \"kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5\") pod \"payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") " pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.513849 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.513689    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume\") pod \"payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") " pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.514412 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.514391    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.514459 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.514430    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume\") pod \"payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") " pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.521923 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.521899    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r85wx\" (UniqueName: \"kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx\") pod \"payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.522067 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.521982    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw4m5\" (UniqueName: \"kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5\") pod \"payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") " pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.605564 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.605474    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:14.642866 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.642806    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:14.924489 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.924462    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"]
Oct 31 01:01:14.926884 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:01:14.926840    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9754c908_f43a_49ef_ad80_0a61344a2864.slice/crio-2fa6083ecb33be1f619d1794c62e56682f00a5c398ace87f6a43fcab247cb417 WatchSource:0}: Error finding container 2fa6083ecb33be1f619d1794c62e56682f00a5c398ace87f6a43fcab247cb417: Status 404 returned error can't find the container with id 2fa6083ecb33be1f619d1794c62e56682f00a5c398ace87f6a43fcab247cb417
Oct 31 01:01:14.959491 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:14.959460    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"]
Oct 31 01:01:15.363253 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.363218    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd" event={"ID":"2138d54e-26fd-49ff-aa2c-ca20a75ccae2","Type":"ContainerStarted","Data":"3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8"}
Oct 31 01:01:15.363723 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.363275    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd" event={"ID":"2138d54e-26fd-49ff-aa2c-ca20a75ccae2","Type":"ContainerStarted","Data":"b23c640a11e8cefa555b5c88dbd57645e564e057be85d203733eb68cb914f413"}
Oct 31 01:01:15.363723 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.363328    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:15.365060 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.365037    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2" event={"ID":"9754c908-f43a-49ef-ad80-0a61344a2864","Type":"ContainerStarted","Data":"e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738"}
Oct 31 01:01:15.365155 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.365084    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2" event={"ID":"9754c908-f43a-49ef-ad80-0a61344a2864","Type":"ContainerStarted","Data":"2fa6083ecb33be1f619d1794c62e56682f00a5c398ace87f6a43fcab247cb417"}
Oct 31 01:01:15.365155 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.365113    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:15.379988 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.379935    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd" podStartSLOduration=1.379917791 podStartE2EDuration="1.379917791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:15.37867713 +0000 UTC m=+1675.749460916" watchObservedRunningTime="2026-10-31 01:01:15.379917791 +0000 UTC m=+1675.750701581"
Oct 31 01:01:15.395961 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:15.395908    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2" podStartSLOduration=1.395893055 podStartE2EDuration="1.395893055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:15.393897405 +0000 UTC m=+1675.764681200" watchObservedRunningTime="2026-10-31 01:01:15.395893055 +0000 UTC m=+1675.766676844"
Oct 31 01:01:23.596325 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.596288    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"]
Oct 31 01:01:23.602441 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.602397    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.609186 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.609066    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:01:23.610472 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.610356    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"]
Oct 31 01:01:23.612047 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.611993    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slc7v\""
Oct 31 01:01:23.749389 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.749343    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.749615 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.749467    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.749615 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.749552    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqvlb\" (UniqueName: \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.851007 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.850914    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.851007 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.850985    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.851278 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.851066    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqvlb\" (UniqueName: \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.852333 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.852300    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.854345 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:23.854320    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:23.861048 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:23.861025    4924 projected.go:196] Error preparing data for projected volume kube-api-access-nqvlb for pod odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl: failed to fetch token: serviceaccounts "maas-api-cleanup" not found
Oct 31 01:01:23.861137 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:23.861110    4924 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb podName:8b6a4604-b534-4eb7-bf3a-85f7a744d0ad nodeName:}" failed. No retries permitted until 2026-10-31 01:01:24.361091972 +0000 UTC m=+1684.731875768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nqvlb" (UniqueName: "kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb") pod "maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" (UID: "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad") : failed to fetch token: serviceaccounts "maas-api-cleanup" not found
Oct 31 01:01:24.458012 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:24.457964    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqvlb\" (UniqueName: \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:24.480660 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:24.480624    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqvlb\" (UniqueName: \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\") pod \"maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:24.534028 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:24.533981    4924 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" secret="" err="secret \"maas-api-cleanup-dockercfg-slc7v\" not found"
Oct 31 01:01:24.534220 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:24.534067    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:24.815222 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:24.815192    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"]
Oct 31 01:01:24.816922 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:01:24.816890    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6a4604_b534_4eb7_bf3a_85f7a744d0ad.slice/crio-3d8e75a7ff2b554d1a99a60e83542cbe285a60b22bc04be028c9aa7c5a094ddc WatchSource:0}: Error finding container 3d8e75a7ff2b554d1a99a60e83542cbe285a60b22bc04be028c9aa7c5a094ddc: Status 404 returned error can't find the container with id 3d8e75a7ff2b554d1a99a60e83542cbe285a60b22bc04be028c9aa7c5a094ddc
Oct 31 01:01:25.446184 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:25.446092    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" event={"ID":"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad","Type":"ContainerStarted","Data":"a039ded5e433355f69e41f8f5ce9c7c3b1369bd51cb533a5a7b39a2201ab22b6"}
Oct 31 01:01:25.446184 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:25.446146    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" event={"ID":"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad","Type":"ContainerStarted","Data":"3d8e75a7ff2b554d1a99a60e83542cbe285a60b22bc04be028c9aa7c5a094ddc"}
Oct 31 01:01:25.446463 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:25.446442    4924 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" secret="" err="secret \"maas-api-cleanup-dockercfg-slc7v\" not found"
Oct 31 01:01:25.477558 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:25.477498    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" podStartSLOduration=2.477483567 podStartE2EDuration="2.477483567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:25.475587132 +0000 UTC m=+1685.846370930" watchObservedRunningTime="2026-10-31 01:01:25.477483567 +0000 UTC m=+1685.848267353"
Oct 31 01:01:26.375958 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.375925    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:26.376380 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.375991    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:26.455323 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.455292    4924 generic.go:356] "Generic (PLEG): container finished" podID="8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" containerID="a039ded5e433355f69e41f8f5ce9c7c3b1369bd51cb533a5a7b39a2201ab22b6" exitCode=0
Oct 31 01:01:26.455502 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.455378    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" event={"ID":"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad","Type":"ContainerDied","Data":"a039ded5e433355f69e41f8f5ce9c7c3b1369bd51cb533a5a7b39a2201ab22b6"}
Oct 31 01:01:26.457324 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.457296    4924 generic.go:356] "Generic (PLEG): container finished" podID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerID="478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983" exitCode=0
Oct 31 01:01:26.457451 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.457368    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" event={"ID":"445e3f60-f5f0-4e4b-b452-c2fd644f638a","Type":"ContainerDied","Data":"478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"}
Oct 31 01:01:26.457891 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.457872    4924 scope.go:122] "RemoveContainer" containerID="478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"
Oct 31 01:01:26.459319 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.459287    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:26.459433 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.459356    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:26.459433 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:26.459399    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:27.343461 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.343425    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:27.465549 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.465513    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" event={"ID":"445e3f60-f5f0-4e4b-b452-c2fd644f638a","Type":"ContainerStarted","Data":"456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2"}
Oct 31 01:01:27.658620 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.658589    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:27.817769 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.817050    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca\") pod \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") "
Oct 31 01:01:27.817769 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.817142    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca\") pod \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") "
Oct 31 01:01:27.817769 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.817212    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\" (UniqueName: \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\") pod \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\" (UID: \"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad\") "
Oct 31 01:01:27.818974 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.818916    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca" pod "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" (UID: "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:27.820680 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.820653    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb" pod "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" (UID: "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad"). InnerVolumeSpecName "kube-api-access-nqvlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:27.821095 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.821067    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca" pod "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" (UID: "8b6a4604-b534-4eb7-bf3a-85f7a744d0ad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:01:27.856027 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:27.855981    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:01:27.926616 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.926512    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:27.926914 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.926890    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:27.927046 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:27.927019    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqvlb\" (UniqueName: \"kubernetes.io/projected/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad-kube-api-access-nqvlb\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:28.145571 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:28.145534    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:28.146613 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:28.146582    4924 patch_prober.go:29] interesting pod/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:01:28.146767 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:28.146641    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 31 01:01:28.472833 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:28.472801    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"
Oct 31 01:01:28.473333 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:28.472838    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl" event={"ID":"8b6a4604-b534-4eb7-bf3a-85f7a744d0ad","Type":"ContainerDied","Data":"3d8e75a7ff2b554d1a99a60e83542cbe285a60b22bc04be028c9aa7c5a094ddc"}
Oct 31 01:01:28.473333 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:28.472887    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d8e75a7ff2b554d1a99a60e83542cbe285a60b22bc04be028c9aa7c5a094ddc"
Oct 31 01:01:29.145834 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:29.145795    4924 patch_prober.go:29] interesting pod/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:01:29.146029 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:29.145853    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 31 01:01:30.145835 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:30.145797    4924 patch_prober.go:29] interesting pod/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:01:30.146257 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:30.145854    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 31 01:01:31.145539 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:31.145500    4924 patch_prober.go:29] interesting pod/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:01:31.145729 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:31.145560    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 31 01:01:32.145822 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:32.145785    4924 patch_prober.go:29] interesting pod/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:01:32.146242 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:32.145844    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 31 01:01:33.149307 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:33.149275    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:33.149713 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:33.149683    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:33.150440 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:33.150421    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:33.702308 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:33.702268    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"]
Oct 31 01:01:33.716526 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:33.716487    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"]
Oct 31 01:01:35.052239 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:35.052188    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"]
Oct 31 01:01:35.058873 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:35.058841    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7e1e1fc6-66qrl"]
Oct 31 01:01:35.763956 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:35.763922    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" path="/var/lib/kubelet/pods/8b6a4604-b534-4eb7-bf3a-85f7a744d0ad/volumes"
Oct 31 01:01:38.703386 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:38.703337    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd" podUID="2138d54e-26fd-49ff-aa2c-ca20a75ccae2" containerName="payload-processing" containerID="cri-o://3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8" gracePeriod=25
Oct 31 01:01:38.717361 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:38.717324    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2" podUID="9754c908-f43a-49ef-ad80-0a61344a2864" containerName="payload-pre-processing" containerID="cri-o://e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738" gracePeriod=25
Oct 31 01:01:39.000293 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.000265    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:39.031973 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.031943    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:39.043890 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.043848    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5\" (UniqueName: \"kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5\") pod \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") "
Oct 31 01:01:39.044072 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.044028    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume\") pod \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\" (UID: \"2138d54e-26fd-49ff-aa2c-ca20a75ccae2\") "
Oct 31 01:01:39.044598 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.044567    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume" pod "2138d54e-26fd-49ff-aa2c-ca20a75ccae2" (UID: "2138d54e-26fd-49ff-aa2c-ca20a75ccae2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:39.046549 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.046517    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5" pod "2138d54e-26fd-49ff-aa2c-ca20a75ccae2" (UID: "2138d54e-26fd-49ff-aa2c-ca20a75ccae2"). InnerVolumeSpecName "kube-api-access-zw4m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:39.145246 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.145213    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume\") pod \"9754c908-f43a-49ef-ad80-0a61344a2864\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") "
Oct 31 01:01:39.145421 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.145313    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx\" (UniqueName: \"kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx\") pod \"9754c908-f43a-49ef-ad80-0a61344a2864\" (UID: \"9754c908-f43a-49ef-ad80-0a61344a2864\") "
Oct 31 01:01:39.145655 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.145634    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:39.145710 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.145661    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw4m5\" (UniqueName: \"kubernetes.io/projected/2138d54e-26fd-49ff-aa2c-ca20a75ccae2-kube-api-access-zw4m5\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:39.145710 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.145675    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume" pod "9754c908-f43a-49ef-ad80-0a61344a2864" (UID: "9754c908-f43a-49ef-ad80-0a61344a2864"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:39.147598 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.147566    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx" pod "9754c908-f43a-49ef-ad80-0a61344a2864" (UID: "9754c908-f43a-49ef-ad80-0a61344a2864"). InnerVolumeSpecName "kube-api-access-r85wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:39.246674 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.246590    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r85wx\" (UniqueName: \"kubernetes.io/projected/9754c908-f43a-49ef-ad80-0a61344a2864-kube-api-access-r85wx\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:39.246674 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.246623    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9754c908-f43a-49ef-ad80-0a61344a2864-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:39.543006 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.542970    4924 generic.go:356] "Generic (PLEG): container finished" podID="2138d54e-26fd-49ff-aa2c-ca20a75ccae2" containerID="3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8" exitCode=0
Oct 31 01:01:39.543210 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.543028    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"
Oct 31 01:01:39.543210 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.543056    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd" event={"ID":"2138d54e-26fd-49ff-aa2c-ca20a75ccae2","Type":"ContainerDied","Data":"3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8"}
Oct 31 01:01:39.543210 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.543113    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd" event={"ID":"2138d54e-26fd-49ff-aa2c-ca20a75ccae2","Type":"ContainerDied","Data":"b23c640a11e8cefa555b5c88dbd57645e564e057be85d203733eb68cb914f413"}
Oct 31 01:01:39.543210 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.543138    4924 scope.go:122] "RemoveContainer" containerID="3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8"
Oct 31 01:01:39.544565 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.544544    4924 generic.go:356] "Generic (PLEG): container finished" podID="9754c908-f43a-49ef-ad80-0a61344a2864" containerID="e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738" exitCode=0
Oct 31 01:01:39.544688 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.544584    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"
Oct 31 01:01:39.544759 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.544706    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2" event={"ID":"9754c908-f43a-49ef-ad80-0a61344a2864","Type":"ContainerDied","Data":"e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738"}
Oct 31 01:01:39.544825 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.544739    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2" event={"ID":"9754c908-f43a-49ef-ad80-0a61344a2864","Type":"ContainerDied","Data":"2fa6083ecb33be1f619d1794c62e56682f00a5c398ace87f6a43fcab247cb417"}
Oct 31 01:01:39.554060 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.554041    4924 scope.go:122] "RemoveContainer" containerID="3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8"
Oct 31 01:01:39.554388 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:39.554367    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8\": container with ID starting with 3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8 not found: ID does not exist" containerID="3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8"
Oct 31 01:01:39.554438 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.554397    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8"} err="failed to get container status \"3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8\": rpc error: code = NotFound desc = could not find container \"3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8\": container with ID starting with 3089254a70ca5b1abdd9bc6a91a9abd4cfa4c2e2bdf5da65dae87dc162f44fa8 not found: ID does not exist"
Oct 31 01:01:39.554438 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.554414    4924 scope.go:122] "RemoveContainer" containerID="e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738"
Oct 31 01:01:39.569371 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.569346    4924 scope.go:122] "RemoveContainer" containerID="e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738"
Oct 31 01:01:39.569691 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:39.569666    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738\": container with ID starting with e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738 not found: ID does not exist" containerID="e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738"
Oct 31 01:01:39.569767 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.569701    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738"} err="failed to get container status \"e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738\": rpc error: code = NotFound desc = could not find container \"e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738\": container with ID starting with e3d4d71c49375240229a691f523c7cea62b4ff639c2e27d6728d7ec138999738 not found: ID does not exist"
Oct 31 01:01:39.570783 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.570760    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"]
Oct 31 01:01:39.576668 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.576637    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7e1e1fc6-557cfcdf79-6vlmd"]
Oct 31 01:01:39.587609 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.587579    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"]
Oct 31 01:01:39.593200 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.593172    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7e1e1fc6-68b7d7f48c-knpc2"]
Oct 31 01:01:39.764808 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.764774    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2138d54e-26fd-49ff-aa2c-ca20a75ccae2" path="/var/lib/kubelet/pods/2138d54e-26fd-49ff-aa2c-ca20a75ccae2/volumes"
Oct 31 01:01:39.765288 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:39.765177    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9754c908-f43a-49ef-ad80-0a61344a2864" path="/var/lib/kubelet/pods/9754c908-f43a-49ef-ad80-0a61344a2864/volumes"
Oct 31 01:01:40.034542 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:40.034504    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"]
Oct 31 01:01:40.034811 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:40.034778    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy" containerID="cri-o://456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2" gracePeriod=30
Oct 31 01:01:41.774808 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.774774    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"]
Oct 31 01:01:41.775210 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775197    4924 cpu_manager.go:414] "containerMap: removing container" podUID="8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" containerName="revoke-keys"
Oct 31 01:01:41.775266 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775211    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" containerName="revoke-keys"
Oct 31 01:01:41.775266 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775235    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9754c908-f43a-49ef-ad80-0a61344a2864" containerName="payload-pre-processing"
Oct 31 01:01:41.775266 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775240    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9754c908-f43a-49ef-ad80-0a61344a2864" containerName="payload-pre-processing"
Oct 31 01:01:41.775266 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775259    4924 cpu_manager.go:414] "containerMap: removing container" podUID="2138d54e-26fd-49ff-aa2c-ca20a75ccae2" containerName="payload-processing"
Oct 31 01:01:41.775266 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775263    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2138d54e-26fd-49ff-aa2c-ca20a75ccae2" containerName="payload-processing"
Oct 31 01:01:41.775431 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775334    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="2138d54e-26fd-49ff-aa2c-ca20a75ccae2" containerName="payload-processing"
Oct 31 01:01:41.775431 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775345    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6a4604-b534-4eb7-bf3a-85f7a744d0ad" containerName="revoke-keys"
Oct 31 01:01:41.775431 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.775353    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9754c908-f43a-49ef-ad80-0a61344a2864" containerName="payload-pre-processing"
Oct 31 01:01:41.778529 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.778506    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.785649 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.785624    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-7c2222-openshift-default-dockercfg-xhbzk\""
Oct 31 01:01:41.792803 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.792771    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"]
Oct 31 01:01:41.872404 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872361    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872587 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872425    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx4d7\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872587 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872519    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872678 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872625    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872678 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872663    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872813 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872793    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872873 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872833    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.872985 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.872957    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.873020 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.873005    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974310 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974261    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974524 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974318    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974524 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974363    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974524 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974389    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974709 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974531    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974709 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974592    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx4d7\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974709 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974644    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974896 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974720    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.974896 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.974807    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.975045 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.975020    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.975115 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.975073    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.975263 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.975231    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.975325 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.975271    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.975466 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.975443    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.976873 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.976844    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.977661 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.977642    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.982484 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.982460    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:41.982840 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:41.982818    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx4d7\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7\") pod \"e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") " pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:42.096398 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.096307    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:42.382105 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.382073    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"]
Oct 31 01:01:42.383719 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:01:42.383682    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5e450f_fe73_4f9e_a335_c5abfe604dfe.slice/crio-29dd49a2a7d17b8eebf5315689f21554188816b1ab2268898d4bf58e336ac74e WatchSource:0}: Error finding container 29dd49a2a7d17b8eebf5315689f21554188816b1ab2268898d4bf58e336ac74e: Status 404 returned error can't find the container with id 29dd49a2a7d17b8eebf5315689f21554188816b1ab2268898d4bf58e336ac74e
Oct 31 01:01:42.386172 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.386141    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:42.386252 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.386209    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:42.386252 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.386238    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:42.568451 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.568410    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" event={"ID":"ed5e450f-fe73-4f9e-a335-c5abfe604dfe","Type":"ContainerStarted","Data":"eccbc623277c16515ab6c0de74e8b7bd9bb7fb21063125c63cb58f84733cb394"}
Oct 31 01:01:42.568656 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.568478    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" event={"ID":"ed5e450f-fe73-4f9e-a335-c5abfe604dfe","Type":"ContainerStarted","Data":"29dd49a2a7d17b8eebf5315689f21554188816b1ab2268898d4bf58e336ac74e"}
Oct 31 01:01:42.599010 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:42.598950    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podStartSLOduration=1.598932391 podStartE2EDuration="1.598932391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:42.597495191 +0000 UTC m=+1702.968278979" watchObservedRunningTime="2026-10-31 01:01:42.598932391 +0000 UTC m=+1702.969716181"
Oct 31 01:01:43.097121 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:43.097083    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:43.102634 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:43.102605    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:43.575323 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:43.575292    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:43.576385 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:43.576363    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:01:45.289296 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.289269    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:45.410980 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.410886    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.410980 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.410953    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.410980 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.410977    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411269 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.410999    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411269 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411030    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411269 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411109    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411269 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411174    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411269 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411221    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:45.411269 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411234    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411574 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411366    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket\") pod \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\" (UID: \"445e3f60-f5f0-4e4b-b452-c2fd644f638a\") "
Oct 31 01:01:45.411574 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411424    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:45.411574 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411445    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:45.411846 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411716    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:45.411846 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411733    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:45.411947 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411857    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.411947 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411878    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.411947 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.411893    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.413436 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.413407    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:45.413545 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.413523    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "kube-api-access-b8hbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:45.413738 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.413721    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:45.413828 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.413728    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo" pod "445e3f60-f5f0-4e4b-b452-c2fd644f638a" (UID: "445e3f60-f5f0-4e4b-b452-c2fd644f638a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:01:45.513207 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.513165    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.513207 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.513201    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8hbs\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-kube-api-access-b8hbs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.513207 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.513211    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.513207 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.513220    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/445e3f60-f5f0-4e4b-b452-c2fd644f638a-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.513500 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.513229    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.513500 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.513238    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/445e3f60-f5f0-4e4b-b452-c2fd644f638a-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:45.589018 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.588979    4924 generic.go:356] "Generic (PLEG): container finished" podID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerID="456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2" exitCode=0
Oct 31 01:01:45.589175 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.589037    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"
Oct 31 01:01:45.589175 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.589065    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" event={"ID":"445e3f60-f5f0-4e4b-b452-c2fd644f638a","Type":"ContainerDied","Data":"456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2"}
Oct 31 01:01:45.589175 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.589146    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr" event={"ID":"445e3f60-f5f0-4e4b-b452-c2fd644f638a","Type":"ContainerDied","Data":"e18a8788344ebbf44d3f727eb15542336ebf6e6f822c5a24f709b9e2f33d48c1"}
Oct 31 01:01:45.589175 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.589169    4924 scope.go:122] "RemoveContainer" containerID="456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2"
Oct 31 01:01:45.599791 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.599764    4924 scope.go:122] "RemoveContainer" containerID="478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"
Oct 31 01:01:45.609685 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.609657    4924 scope.go:122] "RemoveContainer" containerID="456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2"
Oct 31 01:01:45.610079 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:45.610056    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2\": container with ID starting with 456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2 not found: ID does not exist" containerID="456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2"
Oct 31 01:01:45.610152 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.610090    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2"} err="failed to get container status \"456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2\": rpc error: code = NotFound desc = could not find container \"456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2\": container with ID starting with 456c68fce20026ab429c14138051622c7dab7c28b5bf4c40bf59a8b61869e9d2 not found: ID does not exist"
Oct 31 01:01:45.610152 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.610109    4924 scope.go:122] "RemoveContainer" containerID="478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"
Oct 31 01:01:45.610420 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:45.610397    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983\": container with ID starting with 478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983 not found: ID does not exist" containerID="478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"
Oct 31 01:01:45.610474 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.610429    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983"} err="failed to get container status \"478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983\": rpc error: code = NotFound desc = could not find container \"478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983\": container with ID starting with 478fcd24b63c9e1240b6843732cafba24e288c86560a31b13a2a0eaa4886f983 not found: ID does not exist"
Oct 31 01:01:45.612046 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.612020    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"]
Oct 31 01:01:45.616042 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.616020    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7e1e1fc6-openshift-default-5bd78d96df-6sslr"]
Oct 31 01:01:45.764013 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:45.763934    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" path="/var/lib/kubelet/pods/445e3f60-f5f0-4e4b-b452-c2fd644f638a/volumes"
Oct 31 01:01:46.125035 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125001    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"]
Oct 31 01:01:46.125453 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125441    4924 cpu_manager.go:414] "containerMap: removing container" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy"
Oct 31 01:01:46.125495 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125456    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy"
Oct 31 01:01:46.125495 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125479    4924 cpu_manager.go:414] "containerMap: removing container" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy"
Oct 31 01:01:46.125495 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125484    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy"
Oct 31 01:01:46.125605 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125556    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy"
Oct 31 01:01:46.125605 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.125568    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="445e3f60-f5f0-4e4b-b452-c2fd644f638a" containerName="istio-proxy"
Oct 31 01:01:46.130266 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.130241    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.134970 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.134943    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-7c2222\""
Oct 31 01:01:46.138640 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.138622    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-7c2222-dockercfg-lp65m\""
Oct 31 01:01:46.150606 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.150578    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"]
Oct 31 01:01:46.170543 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.170509    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"]
Oct 31 01:01:46.174330 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.174305    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.188216 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.188186    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"]
Oct 31 01:01:46.219940 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.219901    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.220131 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.219966    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2dd6\" (UniqueName: \"kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6\") pod \"payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.321825 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.321722    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whw6c\" (UniqueName: \"kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c\") pod \"payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.322300 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.321974    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.322300 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.322044    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.322300 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.322093    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2dd6\" (UniqueName: \"kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6\") pod \"payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.322983 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.322957    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.330700 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.330669    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2dd6\" (UniqueName: \"kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6\") pod \"payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.423362 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.423265    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.423362 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.423332    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whw6c\" (UniqueName: \"kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c\") pod \"payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.424145 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.424119    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.432841 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.432809    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whw6c\" (UniqueName: \"kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c\") pod \"payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.444493 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.444468    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:46.500606 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.500559    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:46.738226 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.738181    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"]
Oct 31 01:01:46.738701 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:01:46.738667    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad222da_dc30_4de8_b46a_b706da74e01d.slice/crio-ed34952e30d78fb9b27e514cc11249021bf393b42fb1edd33090e0e11952dd77 WatchSource:0}: Error finding container ed34952e30d78fb9b27e514cc11249021bf393b42fb1edd33090e0e11952dd77: Status 404 returned error can't find the container with id ed34952e30d78fb9b27e514cc11249021bf393b42fb1edd33090e0e11952dd77
Oct 31 01:01:46.806852 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:46.806824    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"]
Oct 31 01:01:46.807515 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:01:46.807492    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40102730_f7b1_4f0e_8edb_d2df65cc067c.slice/crio-39fa762283e0f590ba7ef3fea34736f59cf8ca39f97ebc5eb406d8d8f0308f5b WatchSource:0}: Error finding container 39fa762283e0f590ba7ef3fea34736f59cf8ca39f97ebc5eb406d8d8f0308f5b: Status 404 returned error can't find the container with id 39fa762283e0f590ba7ef3fea34736f59cf8ca39f97ebc5eb406d8d8f0308f5b
Oct 31 01:01:47.618284 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.618234    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh" event={"ID":"40102730-f7b1-4f0e-8edb-d2df65cc067c","Type":"ContainerStarted","Data":"9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88"}
Oct 31 01:01:47.618741 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.618304    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh" event={"ID":"40102730-f7b1-4f0e-8edb-d2df65cc067c","Type":"ContainerStarted","Data":"39fa762283e0f590ba7ef3fea34736f59cf8ca39f97ebc5eb406d8d8f0308f5b"}
Oct 31 01:01:47.618741 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.618559    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:47.620088 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.620063    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55" event={"ID":"9ad222da-dc30-4de8-b46a-b706da74e01d","Type":"ContainerStarted","Data":"597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30"}
Oct 31 01:01:47.620189 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.620107    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55" event={"ID":"9ad222da-dc30-4de8-b46a-b706da74e01d","Type":"ContainerStarted","Data":"ed34952e30d78fb9b27e514cc11249021bf393b42fb1edd33090e0e11952dd77"}
Oct 31 01:01:47.620189 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.620139    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:47.636180 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.636125    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh" podStartSLOduration=1.636105957 podStartE2EDuration="1.636105957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:47.63308787 +0000 UTC m=+1708.003871652" watchObservedRunningTime="2026-10-31 01:01:47.636105957 +0000 UTC m=+1708.006889745"
Oct 31 01:01:47.650578 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:47.650516    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55" podStartSLOduration=1.6505014139999998 podStartE2EDuration="1.650501414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:47.648230793 +0000 UTC m=+1708.019014581" watchObservedRunningTime="2026-10-31 01:01:47.650501414 +0000 UTC m=+1708.021285202"
Oct 31 01:01:57.853567 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:01:57.853505    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:01:58.577321 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.577281    4924 patch_prober.go:29] interesting pod/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 31 01:01:58.577552 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.577363    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 31 01:01:58.631046 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.631018    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:01:58.631198 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.631085    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:01:58.695657 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.695616    4924 generic.go:356] "Generic (PLEG): container finished" podID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerID="eccbc623277c16515ab6c0de74e8b7bd9bb7fb21063125c63cb58f84733cb394" exitCode=0
Oct 31 01:01:58.695862 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.695683    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" event={"ID":"ed5e450f-fe73-4f9e-a335-c5abfe604dfe","Type":"ContainerDied","Data":"eccbc623277c16515ab6c0de74e8b7bd9bb7fb21063125c63cb58f84733cb394"}
Oct 31 01:01:58.696131 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.696115    4924 scope.go:122] "RemoveContainer" containerID="eccbc623277c16515ab6c0de74e8b7bd9bb7fb21063125c63cb58f84733cb394"
Oct 31 01:01:58.697593 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.697558    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:58.697675 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.697655    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:58.697713 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:58.697703    4924 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 31 01:01:59.724985 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:01:59.724946    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" event={"ID":"ed5e450f-fe73-4f9e-a335-c5abfe604dfe","Type":"ContainerStarted","Data":"0d4f2d283e5ff3a46f3b730dde442f8c410f987e604cd5a694d5c61641fbcbad"}
Oct 31 01:02:00.097158 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:00.097119    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:02:00.097514 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:00.097491    4924 patch_prober.go:29] interesting pod/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 31 01:02:00.097679 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:00.097655    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 31 01:02:01.101213 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:01.101170    4924 patch_prober.go:29] interesting pod/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 31 01:02:01.101875 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:01.101837    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 31 01:02:02.099263 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:02.099217    4924 patch_prober.go:29] interesting pod/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 31 01:02:02.099584 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:02.099553    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 31 01:02:03.097275 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:03.097228    4924 patch_prober.go:29] interesting pod/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 31 01:02:03.097942 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:03.097909    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 31 01:02:04.115771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:04.114180    4924 patch_prober.go:29] interesting pod/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 31 01:02:04.115771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:04.114250    4924 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 31 01:02:05.100852 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:05.100817    4924 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:02:05.101239 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:05.101216    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:02:05.101830 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:05.101811    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:02:27.638927 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:27.638884    4924 scope.go:122] "RemoveContainer" containerID="eb891b9afb26ff4a4bfcbdd8c2762a6e35c60163136af52d3ec32445654ea4f3"
Oct 31 01:02:27.648626 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:27.648604    4924 scope.go:122] "RemoveContainer" containerID="b030ead98baa4ae0faef0e4efd981c48746d49554ab9ab8a7261228d5132532e"
Oct 31 01:02:27.657543 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:27.657518    4924 scope.go:122] "RemoveContainer" containerID="9ad1c3343a6733ffc2dfa105be1401b615016da42544e36a82cdc6d258e641a9"
Oct 31 01:02:27.852881 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:02:27.852847    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:02:49.027781 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:49.027724    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"]
Oct 31 01:02:49.030267 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:49.030239    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-4zbp4"]
Oct 31 01:02:49.765179 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:02:49.765145    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fae6ec8f-eade-485b-be9c-847dab16bb77" path="/var/lib/kubelet/pods/fae6ec8f-eade-485b-be9c-847dab16bb77/volumes"
Oct 31 01:02:57.853894 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:02:57.853856    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:03:08.029428 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:08.029385    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"]
Oct 31 01:03:08.032028 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:08.032002    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-r4z8q"]
Oct 31 01:03:09.763506 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:09.763473    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df3187b5-cd4b-4ec6-86b3-878648f0d311" path="/var/lib/kubelet/pods/df3187b5-cd4b-4ec6-86b3-878648f0d311/volumes"
Oct 31 01:03:19.846937 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:19.846895    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"]
Oct 31 01:03:19.850759 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:19.850729    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:19.859215 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:19.859186    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:03:19.859402 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:19.859316    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lcclr\""
Oct 31 01:03:19.863554 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:19.863526    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"]
Oct 31 01:03:20.029024 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.028985    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.029024 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.029032    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.029245 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.029213    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vbwz\" (UniqueName: \"kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.130371 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.130274    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.130565 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.130397    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vbwz\" (UniqueName: \"kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.130565 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.130446    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.131047 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.131019    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.132852 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.132831    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.138162 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.138139    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vbwz\" (UniqueName: \"kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz\") pod \"maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.177008 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.176972    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:20.456839 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:20.456803    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"]
Oct 31 01:03:20.458548 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:03:20.458509    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741bd833_0cd0_4aaf_b297_5b60d04be304.slice/crio-f55c785de19f201895a5d6de05627fbeebbd881cae8a91d7f27aaaee44f496d1 WatchSource:0}: Error finding container f55c785de19f201895a5d6de05627fbeebbd881cae8a91d7f27aaaee44f496d1: Status 404 returned error can't find the container with id f55c785de19f201895a5d6de05627fbeebbd881cae8a91d7f27aaaee44f496d1
Oct 31 01:03:21.258687 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:21.258645    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg" event={"ID":"741bd833-0cd0-4aaf-b297-5b60d04be304","Type":"ContainerStarted","Data":"3c3fc6db6a2d4e79f0917e6e4de6fadd75cd2a865ec87fecfe285590dab3ee41"}
Oct 31 01:03:21.259108 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:21.258706    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg" event={"ID":"741bd833-0cd0-4aaf-b297-5b60d04be304","Type":"ContainerStarted","Data":"f55c785de19f201895a5d6de05627fbeebbd881cae8a91d7f27aaaee44f496d1"}
Oct 31 01:03:21.277195 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:21.277144    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg" podStartSLOduration=2.27712984 podStartE2EDuration="2.27712984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:03:21.275284892 +0000 UTC m=+1801.646068681" watchObservedRunningTime="2026-10-31 01:03:21.27712984 +0000 UTC m=+1801.647913628"
Oct 31 01:03:22.266394 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:22.266359    4924 generic.go:356] "Generic (PLEG): container finished" podID="741bd833-0cd0-4aaf-b297-5b60d04be304" containerID="3c3fc6db6a2d4e79f0917e6e4de6fadd75cd2a865ec87fecfe285590dab3ee41" exitCode=0
Oct 31 01:03:22.266856 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:22.266391    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg" event={"ID":"741bd833-0cd0-4aaf-b297-5b60d04be304","Type":"ContainerDied","Data":"3c3fc6db6a2d4e79f0917e6e4de6fadd75cd2a865ec87fecfe285590dab3ee41"}
Oct 31 01:03:23.428279 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.428251    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:23.462222 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.462188    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca\") pod \"741bd833-0cd0-4aaf-b297-5b60d04be304\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") "
Oct 31 01:03:23.462408 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.462234    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca\") pod \"741bd833-0cd0-4aaf-b297-5b60d04be304\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") "
Oct 31 01:03:23.462408 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.462272    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz\" (UniqueName: \"kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz\") pod \"741bd833-0cd0-4aaf-b297-5b60d04be304\" (UID: \"741bd833-0cd0-4aaf-b297-5b60d04be304\") "
Oct 31 01:03:23.462664 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.462636    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca" pod "741bd833-0cd0-4aaf-b297-5b60d04be304" (UID: "741bd833-0cd0-4aaf-b297-5b60d04be304"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:23.464432 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.464408    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz" pod "741bd833-0cd0-4aaf-b297-5b60d04be304" (UID: "741bd833-0cd0-4aaf-b297-5b60d04be304"). InnerVolumeSpecName "kube-api-access-6vbwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:23.464537 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.464428    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca" pod "741bd833-0cd0-4aaf-b297-5b60d04be304" (UID: "741bd833-0cd0-4aaf-b297-5b60d04be304"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:03:23.563305 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.563215    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:23.563305 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.563247    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/741bd833-0cd0-4aaf-b297-5b60d04be304-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:23.563305 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:23.563257    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vbwz\" (UniqueName: \"kubernetes.io/projected/741bd833-0cd0-4aaf-b297-5b60d04be304-kube-api-access-6vbwz\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:24.279569 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:24.279542    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"
Oct 31 01:03:24.279569 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:24.279553    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg" event={"ID":"741bd833-0cd0-4aaf-b297-5b60d04be304","Type":"ContainerDied","Data":"f55c785de19f201895a5d6de05627fbeebbd881cae8a91d7f27aaaee44f496d1"}
Oct 31 01:03:24.279806 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:24.279618    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f55c785de19f201895a5d6de05627fbeebbd881cae8a91d7f27aaaee44f496d1"
Oct 31 01:03:26.503271 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:26.503236    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 01:03:26.511483 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:26.511458    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 01:03:26.791533 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:26.791509    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 01:03:26.799757 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:26.799727    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmwrk_6643e3d6-52ac-4e9d-af24-8a18f92b3748/ovn-acl-logging/0.log"
Oct 31 01:03:27.852907 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:03:27.852871    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:03:29.950036 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:29.949988    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"]
Oct 31 01:03:29.967213 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:29.967172    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"]
Oct 31 01:03:34.951112 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:34.951049    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh" podUID="40102730-f7b1-4f0e-8edb-d2df65cc067c" containerName="payload-processing" containerID="cri-o://9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88" gracePeriod=25
Oct 31 01:03:34.967831 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:34.967798    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55" podUID="9ad222da-dc30-4de8-b46a-b706da74e01d" containerName="payload-pre-processing" containerID="cri-o://597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30" gracePeriod=25
Oct 31 01:03:35.223073 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.223044    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:03:35.263323 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.263290    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:03:35.288288 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.288249    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume\") pod \"40102730-f7b1-4f0e-8edb-d2df65cc067c\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") "
Oct 31 01:03:35.288478 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.288329    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c\" (UniqueName: \"kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c\") pod \"40102730-f7b1-4f0e-8edb-d2df65cc067c\" (UID: \"40102730-f7b1-4f0e-8edb-d2df65cc067c\") "
Oct 31 01:03:35.288720 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.288694    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume" pod "40102730-f7b1-4f0e-8edb-d2df65cc067c" (UID: "40102730-f7b1-4f0e-8edb-d2df65cc067c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:35.290727 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.290684    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c" pod "40102730-f7b1-4f0e-8edb-d2df65cc067c" (UID: "40102730-f7b1-4f0e-8edb-d2df65cc067c"). InnerVolumeSpecName "kube-api-access-whw6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:35.347950 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.347912    4924 generic.go:356] "Generic (PLEG): container finished" podID="9ad222da-dc30-4de8-b46a-b706da74e01d" containerID="597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30" exitCode=0
Oct 31 01:03:35.348152 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.347960    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"
Oct 31 01:03:35.348152 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.348000    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55" event={"ID":"9ad222da-dc30-4de8-b46a-b706da74e01d","Type":"ContainerDied","Data":"597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30"}
Oct 31 01:03:35.348152 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.348051    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55" event={"ID":"9ad222da-dc30-4de8-b46a-b706da74e01d","Type":"ContainerDied","Data":"ed34952e30d78fb9b27e514cc11249021bf393b42fb1edd33090e0e11952dd77"}
Oct 31 01:03:35.348152 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.348079    4924 scope.go:122] "RemoveContainer" containerID="597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30"
Oct 31 01:03:35.349335 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.349314    4924 generic.go:356] "Generic (PLEG): container finished" podID="40102730-f7b1-4f0e-8edb-d2df65cc067c" containerID="9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88" exitCode=0
Oct 31 01:03:35.349497 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.349353    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"
Oct 31 01:03:35.349497 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.349401    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh" event={"ID":"40102730-f7b1-4f0e-8edb-d2df65cc067c","Type":"ContainerDied","Data":"9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88"}
Oct 31 01:03:35.349497 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.349447    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh" event={"ID":"40102730-f7b1-4f0e-8edb-d2df65cc067c","Type":"ContainerDied","Data":"39fa762283e0f590ba7ef3fea34736f59cf8ca39f97ebc5eb406d8d8f0308f5b"}
Oct 31 01:03:35.362860 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.362737    4924 scope.go:122] "RemoveContainer" containerID="597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30"
Oct 31 01:03:35.363238 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:03:35.363211    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30\": container with ID starting with 597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30 not found: ID does not exist" containerID="597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30"
Oct 31 01:03:35.363335 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.363249    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30"} err="failed to get container status \"597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30\": rpc error: code = NotFound desc = could not find container \"597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30\": container with ID starting with 597a534ed257f1d3216124d536a170c7e2e2ec910afd6827a697e65ac8022e30 not found: ID does not exist"
Oct 31 01:03:35.363335 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.363273    4924 scope.go:122] "RemoveContainer" containerID="9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88"
Oct 31 01:03:35.374487 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.374464    4924 scope.go:122] "RemoveContainer" containerID="9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88"
Oct 31 01:03:35.374832 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:03:35.374807    4924 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88\": container with ID starting with 9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88 not found: ID does not exist" containerID="9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88"
Oct 31 01:03:35.374882 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.374841    4924 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88"} err="failed to get container status \"9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88\": rpc error: code = NotFound desc = could not find container \"9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88\": container with ID starting with 9500349a01b424623166d04acfbda653274e3a992a090ef08306e45200c5de88 not found: ID does not exist"
Oct 31 01:03:35.382787 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.380416    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"]
Oct 31 01:03:35.383410 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.383389    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-7c2222-d957f84c-znzdh"]
Oct 31 01:03:35.389357 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.389328    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume\") pod \"9ad222da-dc30-4de8-b46a-b706da74e01d\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") "
Oct 31 01:03:35.389524 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.389396    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6\" (UniqueName: \"kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6\") pod \"9ad222da-dc30-4de8-b46a-b706da74e01d\" (UID: \"9ad222da-dc30-4de8-b46a-b706da74e01d\") "
Oct 31 01:03:35.389781 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.389757    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume" pod "9ad222da-dc30-4de8-b46a-b706da74e01d" (UID: "9ad222da-dc30-4de8-b46a-b706da74e01d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:35.389887 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.389856    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40102730-f7b1-4f0e-8edb-d2df65cc067c-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:35.389887 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.389874    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whw6c\" (UniqueName: \"kubernetes.io/projected/40102730-f7b1-4f0e-8edb-d2df65cc067c-kube-api-access-whw6c\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:35.390005 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.389888    4924 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad222da-dc30-4de8-b46a-b706da74e01d-plugins-config-volume\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:35.391634 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.391611    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6" pod "9ad222da-dc30-4de8-b46a-b706da74e01d" (UID: "9ad222da-dc30-4de8-b46a-b706da74e01d"). InnerVolumeSpecName "kube-api-access-c2dd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:35.449194 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.449141    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"]
Oct 31 01:03:35.451589 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.451563    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-7c2222-x64bg"]
Oct 31 01:03:35.490622 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.490518    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2dd6\" (UniqueName: \"kubernetes.io/projected/9ad222da-dc30-4de8-b46a-b706da74e01d-kube-api-access-c2dd6\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:35.671050 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.671014    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"]
Oct 31 01:03:35.674966 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.674934    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-7c2222-747fc7bf9-frl55"]
Oct 31 01:03:35.763709 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.763624    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40102730-f7b1-4f0e-8edb-d2df65cc067c" path="/var/lib/kubelet/pods/40102730-f7b1-4f0e-8edb-d2df65cc067c/volumes"
Oct 31 01:03:35.764255 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.764228    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="741bd833-0cd0-4aaf-b297-5b60d04be304" path="/var/lib/kubelet/pods/741bd833-0cd0-4aaf-b297-5b60d04be304/volumes"
Oct 31 01:03:35.764735 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:35.764717    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ad222da-dc30-4de8-b46a-b706da74e01d" path="/var/lib/kubelet/pods/9ad222da-dc30-4de8-b46a-b706da74e01d/volumes"
Oct 31 01:03:42.040423 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:42.040388    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"]
Oct 31 01:03:42.040935 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:42.040644    4924 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy" containerID="cri-o://0d4f2d283e5ff3a46f3b730dde442f8c410f987e604cd5a694d5c61641fbcbad" gracePeriod=30
Oct 31 01:03:47.494833 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.494716    4924 generic.go:356] "Generic (PLEG): container finished" podID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerID="0d4f2d283e5ff3a46f3b730dde442f8c410f987e604cd5a694d5c61641fbcbad" exitCode=0
Oct 31 01:03:47.495482 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.495455    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" event={"ID":"ed5e450f-fe73-4f9e-a335-c5abfe604dfe","Type":"ContainerDied","Data":"0d4f2d283e5ff3a46f3b730dde442f8c410f987e604cd5a694d5c61641fbcbad"}
Oct 31 01:03:47.495645 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.495628    4924 scope.go:122] "RemoveContainer" containerID="eccbc623277c16515ab6c0de74e8b7bd9bb7fb21063125c63cb58f84733cb394"
Oct 31 01:03:47.619897 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.619860    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:03:47.775970 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.775932    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776166 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776008    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776166 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776040    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776166 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776111    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776166 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776143    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776395 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776213    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776395 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776245    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776395 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776281    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:47.776395 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776320    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:47.776395 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776357    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776395 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776391    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert\") pod \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\" (UID: \"ed5e450f-fe73-4f9e-a335-c5abfe604dfe\") "
Oct 31 01:03:47.776711 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776633    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:47.776805 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776738    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:47.776805 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776785    4924 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-credential-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.776912 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776804    4924 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-data\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.776912 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776818    4924 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-socket\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.777014 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.776933    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:47.778652 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.778623    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:47.778796 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.778626    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:03:47.778913 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.778890    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:47.779021 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.778943    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7" pod "ed5e450f-fe73-4f9e-a335-c5abfe604dfe" (UID: "ed5e450f-fe73-4f9e-a335-c5abfe604dfe"). InnerVolumeSpecName "kube-api-access-cx4d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:47.877545 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.877504    4924 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-podinfo\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.877545 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.877537    4924 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-envoy\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.877545 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.877547    4924 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-workload-certs\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.877838 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.877556    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx4d7\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-kube-api-access-cx4d7\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.877838 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.877566    4924 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istio-token\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:47.877838 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:47.877574    4924 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed5e450f-fe73-4f9e-a335-c5abfe604dfe-istiod-ca-cert\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:48.502779 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:48.502713    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f" event={"ID":"ed5e450f-fe73-4f9e-a335-c5abfe604dfe","Type":"ContainerDied","Data":"29dd49a2a7d17b8eebf5315689f21554188816b1ab2268898d4bf58e336ac74e"}
Oct 31 01:03:48.502779 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:48.502759    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"
Oct 31 01:03:48.503351 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:48.502799    4924 scope.go:122] "RemoveContainer" containerID="0d4f2d283e5ff3a46f3b730dde442f8c410f987e604cd5a694d5c61641fbcbad"
Oct 31 01:03:48.527677 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:48.527640    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"]
Oct 31 01:03:48.532342 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:48.532315    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-7c2222-openshift-default-8795778c7-bkb8f"]
Oct 31 01:03:49.763304 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:49.763268    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" path="/var/lib/kubelet/pods/ed5e450f-fe73-4f9e-a335-c5abfe604dfe/volumes"
Oct 31 01:03:52.732770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:52.717944    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-p6krc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:03:52.732770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:52.718014    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-p6krc" podUID="d0017f59-4050-462e-b9c7-e45a6cf0562e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:03:54.229823 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.229786    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"]
Oct 31 01:03:54.230237 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230224    4924 cpu_manager.go:414] "containerMap: removing container" podUID="741bd833-0cd0-4aaf-b297-5b60d04be304" containerName="revoke-keys"
Oct 31 01:03:54.230282 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230238    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="741bd833-0cd0-4aaf-b297-5b60d04be304" containerName="revoke-keys"
Oct 31 01:03:54.230282 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230255    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy"
Oct 31 01:03:54.230282 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230260    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy"
Oct 31 01:03:54.230282 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230279    4924 cpu_manager.go:414] "containerMap: removing container" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230285    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230297    4924 cpu_manager.go:414] "containerMap: removing container" podUID="9ad222da-dc30-4de8-b46a-b706da74e01d" containerName="payload-pre-processing"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230303    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ad222da-dc30-4de8-b46a-b706da74e01d" containerName="payload-pre-processing"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230312    4924 cpu_manager.go:414] "containerMap: removing container" podUID="40102730-f7b1-4f0e-8edb-d2df65cc067c" containerName="payload-processing"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230317    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40102730-f7b1-4f0e-8edb-d2df65cc067c" containerName="payload-processing"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230378    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="40102730-f7b1-4f0e-8edb-d2df65cc067c" containerName="payload-processing"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230387    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="741bd833-0cd0-4aaf-b297-5b60d04be304" containerName="revoke-keys"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230395    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230403    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed5e450f-fe73-4f9e-a335-c5abfe604dfe" containerName="istio-proxy"
Oct 31 01:03:54.230409 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.230409    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ad222da-dc30-4de8-b46a-b706da74e01d" containerName="payload-pre-processing"
Oct 31 01:03:54.235106 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.235085    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.238991 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.238951    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lcclr\""
Oct 31 01:03:54.239138 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.238996    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:03:54.240991 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.240967    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"]
Oct 31 01:03:54.282128 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.282090    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.282315 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.282266    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwl79\" (UniqueName: \"kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.282367 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.282347    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.383886 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.383847    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwl79\" (UniqueName: \"kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.384057 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.383904    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.384057 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.383948    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.384573 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.384543    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.386308 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.386288    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.393147 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.393128    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwl79\" (UniqueName: \"kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79\") pod \"maas-api-revoke-sub-f03629db-64rtl\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.549981 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.549948    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:54.826726 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:54.826694    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"]
Oct 31 01:03:54.828506 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:03:54.828478    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9753065_62fd_4739_af38_5a3cdb15314f.slice/crio-86eff631bb3f80141ffdb1c10a789ab0778c2f898d29c80f5a0088fe9b7df60e WatchSource:0}: Error finding container 86eff631bb3f80141ffdb1c10a789ab0778c2f898d29c80f5a0088fe9b7df60e: Status 404 returned error can't find the container with id 86eff631bb3f80141ffdb1c10a789ab0778c2f898d29c80f5a0088fe9b7df60e
Oct 31 01:03:55.609103 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:55.609062    4924 generic.go:356] "Generic (PLEG): container finished" podID="f9753065-62fd-4739-af38-5a3cdb15314f" containerID="0214b8f355f57d0d5bbbe63db896d74e37d2e62f0dde11ee0cf131fe8322fd54" exitCode=0
Oct 31 01:03:55.609520 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:55.609145    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl" event={"ID":"f9753065-62fd-4739-af38-5a3cdb15314f","Type":"ContainerDied","Data":"0214b8f355f57d0d5bbbe63db896d74e37d2e62f0dde11ee0cf131fe8322fd54"}
Oct 31 01:03:55.609520 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:55.609197    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl" event={"ID":"f9753065-62fd-4739-af38-5a3cdb15314f","Type":"ContainerStarted","Data":"86eff631bb3f80141ffdb1c10a789ab0778c2f898d29c80f5a0088fe9b7df60e"}
Oct 31 01:03:56.749449 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.749419    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:56.807351 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.807308    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca\") pod \"f9753065-62fd-4739-af38-5a3cdb15314f\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") "
Oct 31 01:03:56.807351 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.807344    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79\" (UniqueName: \"kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79\") pod \"f9753065-62fd-4739-af38-5a3cdb15314f\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") "
Oct 31 01:03:56.807606 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.807497    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca\") pod \"f9753065-62fd-4739-af38-5a3cdb15314f\" (UID: \"f9753065-62fd-4739-af38-5a3cdb15314f\") "
Oct 31 01:03:56.807883 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.807843    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca" pod "f9753065-62fd-4739-af38-5a3cdb15314f" (UID: "f9753065-62fd-4739-af38-5a3cdb15314f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:56.809626 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.809594    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79" pod "f9753065-62fd-4739-af38-5a3cdb15314f" (UID: "f9753065-62fd-4739-af38-5a3cdb15314f"). InnerVolumeSpecName "kube-api-access-wwl79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:56.809715 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.809650    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca" pod "f9753065-62fd-4739-af38-5a3cdb15314f" (UID: "f9753065-62fd-4739-af38-5a3cdb15314f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:03:56.908651 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.908570    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:56.908651 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.908597    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f9753065-62fd-4739-af38-5a3cdb15314f-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:56.908651 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:56.908607    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwl79\" (UniqueName: \"kubernetes.io/projected/f9753065-62fd-4739-af38-5a3cdb15314f-kube-api-access-wwl79\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:57.622101 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:57.622069    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl"
Oct 31 01:03:57.622257 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:57.622100    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-64rtl" event={"ID":"f9753065-62fd-4739-af38-5a3cdb15314f","Type":"ContainerDied","Data":"86eff631bb3f80141ffdb1c10a789ab0778c2f898d29c80f5a0088fe9b7df60e"}
Oct 31 01:03:57.622257 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:03:57.622144    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86eff631bb3f80141ffdb1c10a789ab0778c2f898d29c80f5a0088fe9b7df60e"
Oct 31 01:03:57.852837 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:03:57.852800    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:04:10.369427 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.369345    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"]
Oct 31 01:04:10.369973 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.369840    4924 cpu_manager.go:414] "containerMap: removing container" podUID="f9753065-62fd-4739-af38-5a3cdb15314f" containerName="revoke-keys"
Oct 31 01:04:10.369973 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.369853    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9753065-62fd-4739-af38-5a3cdb15314f" containerName="revoke-keys"
Oct 31 01:04:10.369973 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.369930    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9753065-62fd-4739-af38-5a3cdb15314f" containerName="revoke-keys"
Oct 31 01:04:10.373600 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.373574    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.378520 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.378495    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:04:10.378687 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.378589    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lcclr\""
Oct 31 01:04:10.382788 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.382137    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"]
Oct 31 01:04:10.486221 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.486175    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxrvh\" (UniqueName: \"kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.486221 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.486228    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.486485 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.486384    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.587279 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.587234    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.587500 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.587351    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxrvh\" (UniqueName: \"kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.587500 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.587380    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.588134 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.588099    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.589639 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.589615    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.596299 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.596277    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxrvh\" (UniqueName: \"kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh\") pod \"maas-api-revoke-sub-0165d7d2-gc9qs\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.689300 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.689212    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:10.969991 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:10.969956    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"]
Oct 31 01:04:10.971478 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:04:10.971450    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfec5bcb_0ba5_452d_93f2_07370ac7ba31.slice/crio-7de9cd2b6ebfba5ce7497572091dc6c79f4a58829ca4b33996e105ca66b0bc06 WatchSource:0}: Error finding container 7de9cd2b6ebfba5ce7497572091dc6c79f4a58829ca4b33996e105ca66b0bc06: Status 404 returned error can't find the container with id 7de9cd2b6ebfba5ce7497572091dc6c79f4a58829ca4b33996e105ca66b0bc06
Oct 31 01:04:11.748002 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:11.747918    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs" event={"ID":"cfec5bcb-0ba5-452d-93f2-07370ac7ba31","Type":"ContainerStarted","Data":"584d772e5e91f0f3849c2b4941647224bbd55709cf562c05b78751f41e142053"}
Oct 31 01:04:11.748002 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:11.747968    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs" event={"ID":"cfec5bcb-0ba5-452d-93f2-07370ac7ba31","Type":"ContainerStarted","Data":"7de9cd2b6ebfba5ce7497572091dc6c79f4a58829ca4b33996e105ca66b0bc06"}
Oct 31 01:04:11.770072 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:11.770024    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs" podStartSLOduration=1.770012162 podStartE2EDuration="1.770012162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:11.768393845 +0000 UTC m=+1852.139177654" watchObservedRunningTime="2026-10-31 01:04:11.770012162 +0000 UTC m=+1852.140795951"
Oct 31 01:04:12.756047 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:12.756010    4924 generic.go:356] "Generic (PLEG): container finished" podID="cfec5bcb-0ba5-452d-93f2-07370ac7ba31" containerID="584d772e5e91f0f3849c2b4941647224bbd55709cf562c05b78751f41e142053" exitCode=0
Oct 31 01:04:12.756463 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:12.756104    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs" event={"ID":"cfec5bcb-0ba5-452d-93f2-07370ac7ba31","Type":"ContainerDied","Data":"584d772e5e91f0f3849c2b4941647224bbd55709cf562c05b78751f41e142053"}
Oct 31 01:04:13.903698 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.903672    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:13.920109 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.920082    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca\") pod \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") "
Oct 31 01:04:13.920267 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.920189    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca\") pod \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") "
Oct 31 01:04:13.920332 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.920292    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh\" (UniqueName: \"kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh\") pod \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\" (UID: \"cfec5bcb-0ba5-452d-93f2-07370ac7ba31\") "
Oct 31 01:04:13.921583 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.921540    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca" pod "cfec5bcb-0ba5-452d-93f2-07370ac7ba31" (UID: "cfec5bcb-0ba5-452d-93f2-07370ac7ba31"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:13.922851 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.922824    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca" pod "cfec5bcb-0ba5-452d-93f2-07370ac7ba31" (UID: "cfec5bcb-0ba5-452d-93f2-07370ac7ba31"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:04:13.923480 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:13.923424    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh" pod "cfec5bcb-0ba5-452d-93f2-07370ac7ba31" (UID: "cfec5bcb-0ba5-452d-93f2-07370ac7ba31"). InnerVolumeSpecName "kube-api-access-dxrvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:14.021293 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:14.021263    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:14.021293 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:14.021295    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:14.021495 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:14.021306    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxrvh\" (UniqueName: \"kubernetes.io/projected/cfec5bcb-0ba5-452d-93f2-07370ac7ba31-kube-api-access-dxrvh\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:14.769828 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:14.769793    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs"
Oct 31 01:04:14.769828 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:14.769810    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-gc9qs" event={"ID":"cfec5bcb-0ba5-452d-93f2-07370ac7ba31","Type":"ContainerDied","Data":"7de9cd2b6ebfba5ce7497572091dc6c79f4a58829ca4b33996e105ca66b0bc06"}
Oct 31 01:04:14.770041 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:14.769858    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7de9cd2b6ebfba5ce7497572091dc6c79f4a58829ca4b33996e105ca66b0bc06"
Oct 31 01:04:21.087617 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.087580    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"]
Oct 31 01:04:21.088205 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.088185    4924 cpu_manager.go:414] "containerMap: removing container" podUID="cfec5bcb-0ba5-452d-93f2-07370ac7ba31" containerName="revoke-keys"
Oct 31 01:04:21.088310 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.088208    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfec5bcb-0ba5-452d-93f2-07370ac7ba31" containerName="revoke-keys"
Oct 31 01:04:21.088397 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.088383    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfec5bcb-0ba5-452d-93f2-07370ac7ba31" containerName="revoke-keys"
Oct 31 01:04:21.091215 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.091193    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.095612 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.095587    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:04:21.096631 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.096596    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lcclr\""
Oct 31 01:04:21.102017 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.101992    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"]
Oct 31 01:04:21.192154 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.192121    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqsjc\" (UniqueName: \"kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.192326 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.192188    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.192326 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.192299    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.292987 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.292952    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.293190 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.292996    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqsjc\" (UniqueName: \"kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.293190 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.293047    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.293806 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.293779    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.295465 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.295438    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.300611 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.300590    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqsjc\" (UniqueName: \"kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc\") pod \"maas-api-revoke-sub-6b19338f-7l2n4\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.411052 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.410954    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:21.705210 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.692318    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"]
Oct 31 01:04:21.705616 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:04:21.705563    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79be0e32_b62f_46b0_9a2f_c6bdec777447.slice/crio-aede469bea83009257daf9e7a933d42cf11b530b8805a12f026945e1227cfabb WatchSource:0}: Error finding container aede469bea83009257daf9e7a933d42cf11b530b8805a12f026945e1227cfabb: Status 404 returned error can't find the container with id aede469bea83009257daf9e7a933d42cf11b530b8805a12f026945e1227cfabb
Oct 31 01:04:21.811421 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:21.811382    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4" event={"ID":"79be0e32-b62f-46b0-9a2f-c6bdec777447","Type":"ContainerStarted","Data":"aede469bea83009257daf9e7a933d42cf11b530b8805a12f026945e1227cfabb"}
Oct 31 01:04:22.820072 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:22.820039    4924 generic.go:356] "Generic (PLEG): container finished" podID="79be0e32-b62f-46b0-9a2f-c6bdec777447" containerID="a03b39e5ffaf3b4aa8907cef748e1e85c4e11fa19c21c2695b801601cb6f3bb7" exitCode=0
Oct 31 01:04:22.820482 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:22.820121    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4" event={"ID":"79be0e32-b62f-46b0-9a2f-c6bdec777447","Type":"ContainerDied","Data":"a03b39e5ffaf3b4aa8907cef748e1e85c4e11fa19c21c2695b801601cb6f3bb7"}
Oct 31 01:04:23.963624 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:23.963597    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:24.028219 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.028170    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc\" (UniqueName: \"kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc\") pod \"79be0e32-b62f-46b0-9a2f-c6bdec777447\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") "
Oct 31 01:04:24.028434 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.028242    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca\") pod \"79be0e32-b62f-46b0-9a2f-c6bdec777447\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") "
Oct 31 01:04:24.028434 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.028410    4924 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca\") pod \"79be0e32-b62f-46b0-9a2f-c6bdec777447\" (UID: \"79be0e32-b62f-46b0-9a2f-c6bdec777447\") "
Oct 31 01:04:24.028842 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.028815    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca" pod "79be0e32-b62f-46b0-9a2f-c6bdec777447" (UID: "79be0e32-b62f-46b0-9a2f-c6bdec777447"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:24.030436 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.030414    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc" pod "79be0e32-b62f-46b0-9a2f-c6bdec777447" (UID: "79be0e32-b62f-46b0-9a2f-c6bdec777447"). InnerVolumeSpecName "kube-api-access-zqsjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:24.030529 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.030455    4924 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca" pod "79be0e32-b62f-46b0-9a2f-c6bdec777447" (UID: "79be0e32-b62f-46b0-9a2f-c6bdec777447"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:04:24.129810 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.129698    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-openshift-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:24.129810 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.129731    4924 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqsjc\" (UniqueName: \"kubernetes.io/projected/79be0e32-b62f-46b0-9a2f-c6bdec777447-kube-api-access-zqsjc\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:24.129810 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.129771    4924 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79be0e32-b62f-46b0-9a2f-c6bdec777447-maas-api-xks-ca\") on node \"ip-10-0-142-217.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:24.833415 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.833385    4924 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4"
Oct 31 01:04:24.833571 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.833425    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-7l2n4" event={"ID":"79be0e32-b62f-46b0-9a2f-c6bdec777447","Type":"ContainerDied","Data":"aede469bea83009257daf9e7a933d42cf11b530b8805a12f026945e1227cfabb"}
Oct 31 01:04:24.833571 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:24.833473    4924 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aede469bea83009257daf9e7a933d42cf11b530b8805a12f026945e1227cfabb"
Oct 31 01:04:27.719573 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:27.719538    4924 scope.go:122] "RemoveContainer" containerID="65edf4e3f28b87eb068534583c4f0497f8753c947d3bfc69d00951bfd83c461c"
Oct 31 01:04:27.728963 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:27.728943    4924 scope.go:122] "RemoveContainer" containerID="3ac0fbea4ff031c47434a743e13f71ac2e448bb3372ae59518f60181448e337e"
Oct 31 01:04:27.853364 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:04:27.853326    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:04:32.723771 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:32.718991    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-759d7ff9c9-vf8p2_7f05576a-ca0d-4e6f-9f10-382a47312af2/authorino/0.log"
Oct 31 01:04:33.956037 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:33.955992    4924 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6f54779779-4f7pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:04:33.956651 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:33.956617    4924 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6f54779779-4f7pj" podUID="fc1366be-1d30-4c52-a330-4164ddae0391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:04:44.043357 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:44.043321    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5bb95b6d85-g6jkp_a41765f7-69c2-45e7-bd3e-93bcc31908a1/manager/0.log"
Oct 31 01:04:46.677442 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:46.677406    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-gc9qs_cfec5bcb-0ba5-452d-93f2-07370ac7ba31/revoke-keys/0.log"
Oct 31 01:04:46.850599 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:46.850565    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-28b214c4-gs4zq_acb199a7-62f3-4139-89ae-8d0f831c2085/revoke-keys/0.log"
Oct 31 01:04:47.034328 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:47.034299    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-x9cqz_613f15ee-fa77-4623-b3a6-1ede827bd225/revoke-keys/0.log"
Oct 31 01:04:47.377836 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:47.377737    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-7l2n4_79be0e32-b62f-46b0-9a2f-c6bdec777447/revoke-keys/0.log"
Oct 31 01:04:47.538238 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:47.538204    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-64rtl_f9753065-62fd-4739-af38-5a3cdb15314f/revoke-keys/0.log"
Oct 31 01:04:47.731267 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:47.731175    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-8gmlk_d69c98ac-3b05-4310-b563-6822a1985b8a/postgres/0.log"
Oct 31 01:04:49.263629 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:49.263596    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-759d7ff9c9-vf8p2_7f05576a-ca0d-4e6f-9f10-382a47312af2/authorino/0.log"
Oct 31 01:04:49.491487 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:49.491457    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-pn9q9_0d165765-1777-4387-952b-1b568c10a0f7/manager/0.log"
Oct 31 01:04:49.666601 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:49.666568    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-ppnqs_3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08/manager/0.log"
Oct 31 01:04:50.065340 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:50.065310    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-hwnxg_d7d12734-f949-4114-b639-543ab45e1f2d/manager/0.log"
Oct 31 01:04:50.475696 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:50.475602    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-hgr7k_48905ee6-4e74-4eb8-b8f8-061f475facdf/manager/0.log"
Oct 31 01:04:51.254080 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:51.254042    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9_2089a27d-9052-4ead-a597-b014675e2de3/istio-proxy/0.log"
Oct 31 01:04:52.181286 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:52.181249    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/1.log"
Oct 31 01:04:52.560831 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:52.560803    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-9c7vk_9495ab3a-0b84-4854-bc5e-30d46c9f9040/istio-proxy/0.log"
Oct 31 01:04:53.318202 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:53.318153    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-p6krc_d0017f59-4050-462e-b9c7-e45a6cf0562e/istio-proxy/0.log"
Oct 31 01:04:53.507870 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:53.507831    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-pmzpl_fc4b4307-477d-44cd-a66a-66eb109867be/istio-proxy/0.log"
Oct 31 01:04:53.863148 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:53.863118    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6_95d1f317-798a-49cd-8dbe-1af184a58719/istio-proxy/0.log"
Oct 31 01:04:54.055852 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:54.055825    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-njw55_ac7e12ef-96ee-46bf-8813-36d45d5e9166/payload-pre-processing/0.log"
Oct 31 01:04:54.232773 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:54.232661    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:04:54.435299 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:54.435268    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d4b7bcd7-2dvsg_3570c076-c980-454d-9b4d-4fbcc88953e0/router/0.log"
Oct 31 01:04:54.877203 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:54.877167    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/0.log"
Oct 31 01:04:55.649597 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:55.649545    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-p6qm4_42092e39-d9c6-47f8-b6dd-b4921bdc955b/main/0.log"
Oct 31 01:04:56.177704 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:56.177668    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk_af9d788e-efd8-48d9-94db-cc87b84bb703/main/0.log"
Oct 31 01:04:56.693977 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:56.693940    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t_581a4f76-b7c1-4293-9b06-1079af9b9e3b/main/0.log"
Oct 31 01:04:57.620068 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:04:57.620032    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-759d7ff9c9-vf8p2_7f05576a-ca0d-4e6f-9f10-382a47312af2/authorino/0.log"
Oct 31 01:04:57.853318 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:04:57.853276    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:05:06.738250 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:06.738205    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5bb95b6d85-g6jkp_a41765f7-69c2-45e7-bd3e-93bcc31908a1/manager/0.log"
Oct 31 01:05:09.287856 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:09.287822    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-gc9qs_cfec5bcb-0ba5-452d-93f2-07370ac7ba31/revoke-keys/0.log"
Oct 31 01:05:09.478389 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:09.478359    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-28b214c4-gs4zq_acb199a7-62f3-4139-89ae-8d0f831c2085/revoke-keys/0.log"
Oct 31 01:05:09.666352 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:09.666272    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-x9cqz_613f15ee-fa77-4623-b3a6-1ede827bd225/revoke-keys/0.log"
Oct 31 01:05:10.030203 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:10.030175    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-7l2n4_79be0e32-b62f-46b0-9a2f-c6bdec777447/revoke-keys/0.log"
Oct 31 01:05:10.214441 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:10.214406    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-64rtl_f9753065-62fd-4739-af38-5a3cdb15314f/revoke-keys/0.log"
Oct 31 01:05:10.427867 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:10.427782    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-8gmlk_d69c98ac-3b05-4310-b563-6822a1985b8a/postgres/0.log"
Oct 31 01:05:11.576004 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:11.575971    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-759d7ff9c9-vf8p2_7f05576a-ca0d-4e6f-9f10-382a47312af2/authorino/0.log"
Oct 31 01:05:11.800345 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:11.800312    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-pn9q9_0d165765-1777-4387-952b-1b568c10a0f7/manager/0.log"
Oct 31 01:05:12.009818 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:12.009699    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-ppnqs_3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08/manager/0.log"
Oct 31 01:05:12.440475 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:12.440443    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-hwnxg_d7d12734-f949-4114-b639-543ab45e1f2d/manager/0.log"
Oct 31 01:05:12.873264 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:12.873227    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-hgr7k_48905ee6-4e74-4eb8-b8f8-061f475facdf/manager/0.log"
Oct 31 01:05:13.450422 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:13.450389    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9_2089a27d-9052-4ead-a597-b014675e2de3/istio-proxy/0.log"
Oct 31 01:05:14.465682 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:14.465639    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/1.log"
Oct 31 01:05:14.890526 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:14.890498    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-9c7vk_9495ab3a-0b84-4854-bc5e-30d46c9f9040/istio-proxy/0.log"
Oct 31 01:05:15.682571 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:15.682535    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-p6krc_d0017f59-4050-462e-b9c7-e45a6cf0562e/istio-proxy/0.log"
Oct 31 01:05:15.923159 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:15.923126    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-pmzpl_fc4b4307-477d-44cd-a66a-66eb109867be/istio-proxy/0.log"
Oct 31 01:05:16.317617 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:16.317583    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6_95d1f317-798a-49cd-8dbe-1af184a58719/istio-proxy/0.log"
Oct 31 01:05:16.515877 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:16.515844    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-njw55_ac7e12ef-96ee-46bf-8813-36d45d5e9166/payload-pre-processing/0.log"
Oct 31 01:05:16.695981 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:16.695889    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:05:16.879612 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:16.879579    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d4b7bcd7-2dvsg_3570c076-c980-454d-9b4d-4fbcc88953e0/router/0.log"
Oct 31 01:05:17.609590 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:17.609558    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-p6qm4_42092e39-d9c6-47f8-b6dd-b4921bdc955b/main/0.log"
Oct 31 01:05:18.177849 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:18.177816    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8twpdk_af9d788e-efd8-48d9-94db-cc87b84bb703/main/0.log"
Oct 31 01:05:18.743060 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:18.743031    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-krq4t_581a4f76-b7c1-4293-9b06-1079af9b9e3b/main/0.log"
Oct 31 01:05:27.854115 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:05:27.854075    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:05:35.029121 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:35.029035    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"]
Oct 31 01:05:35.031484 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:35.031456    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-x9cqz"]
Oct 31 01:05:35.762665 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:35.762621    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="613f15ee-fa77-4623-b3a6-1ede827bd225" path="/var/lib/kubelet/pods/613f15ee-fa77-4623-b3a6-1ede827bd225/volumes"
Oct 31 01:05:54.052963 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:54.052926    4924 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"]
Oct 31 01:05:54.063204 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:54.063169    4924 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28b214c4-gs4zq"]
Oct 31 01:05:55.763907 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:55.763866    4924 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acb199a7-62f3-4139-89ae-8d0f831c2085" path="/var/lib/kubelet/pods/acb199a7-62f3-4139-89ae-8d0f831c2085/volumes"
Oct 31 01:05:57.325666 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:57.325600    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:05:57.515094 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:05:57.515061    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:05:57.851998 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:05:57.851955    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:06:27.852606 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:06:27.852563    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:06:44.170486 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.170450    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:06:44.368208 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.368172    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:06:44.674697 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.674658    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bd47j/must-gather-q2tv4"]
Oct 31 01:06:44.675135 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.675121    4924 cpu_manager.go:414] "containerMap: removing container" podUID="79be0e32-b62f-46b0-9a2f-c6bdec777447" containerName="revoke-keys"
Oct 31 01:06:44.675182 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.675135    4924 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79be0e32-b62f-46b0-9a2f-c6bdec777447" containerName="revoke-keys"
Oct 31 01:06:44.675218 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.675209    4924 memory_manager.go:365] "RemoveStaleState removing state" podUID="79be0e32-b62f-46b0-9a2f-c6bdec777447" containerName="revoke-keys"
Oct 31 01:06:44.678438 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.678418    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.681919 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.681895    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bd47j\"/\"kube-root-ca.crt\""
Oct 31 01:06:44.682841 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.682819    4924 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bd47j\"/\"default-dockercfg-zkc7m\""
Oct 31 01:06:44.682841 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.682824    4924 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bd47j\"/\"openshift-service-ca.crt\""
Oct 31 01:06:44.695785 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.695757    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bd47j/must-gather-q2tv4"]
Oct 31 01:06:44.766311 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.766277    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/654cf55f-258e-4d6e-9248-1bfec13f25cb-must-gather-output\") pod \"must-gather-q2tv4\" (UID: \"654cf55f-258e-4d6e-9248-1bfec13f25cb\") " pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.766490 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.766317    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k7s7\" (UniqueName: \"kubernetes.io/projected/654cf55f-258e-4d6e-9248-1bfec13f25cb-kube-api-access-9k7s7\") pod \"must-gather-q2tv4\" (UID: \"654cf55f-258e-4d6e-9248-1bfec13f25cb\") " pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.867106 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.867069    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/654cf55f-258e-4d6e-9248-1bfec13f25cb-must-gather-output\") pod \"must-gather-q2tv4\" (UID: \"654cf55f-258e-4d6e-9248-1bfec13f25cb\") " pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.867106 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.867111    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k7s7\" (UniqueName: \"kubernetes.io/projected/654cf55f-258e-4d6e-9248-1bfec13f25cb-kube-api-access-9k7s7\") pod \"must-gather-q2tv4\" (UID: \"654cf55f-258e-4d6e-9248-1bfec13f25cb\") " pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.867440 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.867419    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/654cf55f-258e-4d6e-9248-1bfec13f25cb-must-gather-output\") pod \"must-gather-q2tv4\" (UID: \"654cf55f-258e-4d6e-9248-1bfec13f25cb\") " pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.876060 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.876033    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k7s7\" (UniqueName: \"kubernetes.io/projected/654cf55f-258e-4d6e-9248-1bfec13f25cb-kube-api-access-9k7s7\") pod \"must-gather-q2tv4\" (UID: \"654cf55f-258e-4d6e-9248-1bfec13f25cb\") " pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:44.993566 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:44.993481    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bd47j/must-gather-q2tv4"
Oct 31 01:06:45.279472 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:45.277623    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bd47j/must-gather-q2tv4"]
Oct 31 01:06:45.281533 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:06:45.281505    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod654cf55f_258e_4d6e_9248_1bfec13f25cb.slice/crio-9f511dfb6e9583d409a051c8a0ea523e740465bdc90d436a0699dbbb0f91c10f WatchSource:0}: Error finding container 9f511dfb6e9583d409a051c8a0ea523e740465bdc90d436a0699dbbb0f91c10f: Status 404 returned error can't find the container with id 9f511dfb6e9583d409a051c8a0ea523e740465bdc90d436a0699dbbb0f91c10f
Oct 31 01:06:45.283757 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:45.283722    4924 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:06:45.702490 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:45.702400    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bd47j/must-gather-q2tv4" event={"ID":"654cf55f-258e-4d6e-9248-1bfec13f25cb","Type":"ContainerStarted","Data":"9f511dfb6e9583d409a051c8a0ea523e740465bdc90d436a0699dbbb0f91c10f"}
Oct 31 01:06:46.710240 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:46.710204    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bd47j/must-gather-q2tv4" event={"ID":"654cf55f-258e-4d6e-9248-1bfec13f25cb","Type":"ContainerStarted","Data":"c6f3002367587f6033b1b195f27eaf55ac66d9dd99a9de65b9348f3df5777818"}
Oct 31 01:06:46.710640 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:46.710252    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bd47j/must-gather-q2tv4" event={"ID":"654cf55f-258e-4d6e-9248-1bfec13f25cb","Type":"ContainerStarted","Data":"6ea3f543a67bd17d3dbdb8f9b3f2ed748f020a8220668637ed7fe8910e7be88a"}
Oct 31 01:06:46.731023 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:46.730969    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-bd47j/must-gather-q2tv4" podStartSLOduration=1.8724176529999998 podStartE2EDuration="2.730953621s" totalImagesPullingTime="858.535968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:06:46.727659912 +0000 UTC m=+2007.098443698" watchObservedRunningTime="2026-10-31 01:06:46.730953621 +0000 UTC m=+2007.101737414"
Oct 31 01:06:47.954756 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:47.954714    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6gbrr_4c562d34-6b68-4800-a5a6-5c9e204a6050/global-pull-secret-syncer/0.log"
Oct 31 01:06:48.085984 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:48.085957    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-55fsb_995d046f-21e7-4152-bdf2-d0954c8ff37f/connection-checker/0.log"
Oct 31 01:06:48.113376 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:48.113351    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-8dbtk_65327963-1791-42b7-a9a7-4e77b700ed5a/connection-checker/0.log"
Oct 31 01:06:48.139569 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:48.139539    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-llgrf_ab123256-37b2-4cf7-961b-38c7a0e3be21/connection-checker/0.log"
Oct 31 01:06:48.222382 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:48.222258    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hrtzv_542f4323-b31e-4491-9b6e-02a1916798ab/konnectivity-agent/0.log"
Oct 31 01:06:48.293786 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:48.293754    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-217.us-east-2.compute.internal_ece4c01c041ef372ae3784c8037bb134/haproxy/0.log"
Oct 31 01:06:55.683767 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:55.683195    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-759d7ff9c9-vf8p2_7f05576a-ca0d-4e6f-9f10-382a47312af2/authorino/0.log"
Oct 31 01:06:55.725705 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:55.722647    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-pn9q9_0d165765-1777-4387-952b-1b568c10a0f7/manager/0.log"
Oct 31 01:06:55.750966 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:55.750930    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-ppnqs_3f9dfbaa-8253-40c8-ad31-c4fd7ea6fc08/manager/0.log"
Oct 31 01:06:55.875723 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:55.875456    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-hwnxg_d7d12734-f949-4114-b639-543ab45e1f2d/manager/0.log"
Oct 31 01:06:56.051729 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:56.048266    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-hgr7k_48905ee6-4e74-4eb8-b8f8-061f475facdf/manager/0.log"
Oct 31 01:06:57.563826 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:57.563786    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-nsxqz_1dabbb8b-9e70-4c1c-92d7-04d112150395/cluster-monitoring-operator/0.log"
Oct 31 01:06:57.858392 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:57.858307    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nwcch_8f939d23-82c9-43f0-be5f-4799bba43a7c/node-exporter/0.log"
Oct 31 01:06:57.861250 ip-10-0-142-217 kubenswrapper[4924]: E1031 01:06:57.861224    4924 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bcmv5" containerName="extractor"
Oct 31 01:06:57.884461 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:57.884390    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nwcch_8f939d23-82c9-43f0-be5f-4799bba43a7c/kube-rbac-proxy/0.log"
Oct 31 01:06:57.907696 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:57.907671    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nwcch_8f939d23-82c9-43f0-be5f-4799bba43a7c/init-textfile/0.log"
Oct 31 01:06:58.343857 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.343818    4924 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"]
Oct 31 01:06:58.351040 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.349686    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"]
Oct 31 01:06:58.351040 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.349842    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.471406 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.471373    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-podres\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.471588 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.471428    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-lib-modules\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.471588 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.471528    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-sys\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.471588 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.471553    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-proc\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.471780 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.471727    4924 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f4qd\" (UniqueName: \"kubernetes.io/projected/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-kube-api-access-9f4qd\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.573432 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.573386    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f4qd\" (UniqueName: \"kubernetes.io/projected/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-kube-api-access-9f4qd\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574072 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574044    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-podres\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574247 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574230    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-lib-modules\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574378 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574350    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-lib-modules\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574562 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574371    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-sys\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574668 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574580    4924 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-proc\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574815 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574795    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-proc\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574901 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574433    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-sys\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.574901 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.574867    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-podres\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.588271 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.588231    4924 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f4qd\" (UniqueName: \"kubernetes.io/projected/98cceafc-d8fe-46e2-bd8d-eb75c556ba02-kube-api-access-9f4qd\") pod \"perf-node-gather-daemonset-hpzdv\" (UID: \"98cceafc-d8fe-46e2-bd8d-eb75c556ba02\") " pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.684895 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.684806    4924 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:58.972819 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:58.970640    4924 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"]
Oct 31 01:06:58.988868 ip-10-0-142-217 kubenswrapper[4924]: W1031 01:06:58.988820    4924 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98cceafc_d8fe_46e2_bd8d_eb75c556ba02.slice/crio-6adea19922ce5ef7ea325bececa006464bba5adbf15f2d12007b27fd30b39405 WatchSource:0}: Error finding container 6adea19922ce5ef7ea325bececa006464bba5adbf15f2d12007b27fd30b39405: Status 404 returned error can't find the container with id 6adea19922ce5ef7ea325bececa006464bba5adbf15f2d12007b27fd30b39405
Oct 31 01:06:59.686571 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:59.686465    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-v9cnd_f4e15376-708a-442b-9439-8bec92eae34c/networking-console-plugin/0.log"
Oct 31 01:06:59.893457 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:59.893424    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv" event={"ID":"98cceafc-d8fe-46e2-bd8d-eb75c556ba02","Type":"ContainerStarted","Data":"65077d1b6d694e101bd6503f982c8a96dea08aa5025e71ee529b1798c2622c84"}
Oct 31 01:06:59.893654 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:59.893475    4924 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv" event={"ID":"98cceafc-d8fe-46e2-bd8d-eb75c556ba02","Type":"ContainerStarted","Data":"6adea19922ce5ef7ea325bececa006464bba5adbf15f2d12007b27fd30b39405"}
Oct 31 01:06:59.893654 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:59.893538    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:06:59.915770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:06:59.912877    4924 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv" podStartSLOduration=1.9128567269999999 podStartE2EDuration="1.912856727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 01:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:06:59.911679601 +0000 UTC m=+2020.282463392" watchObservedRunningTime="2026-10-31 01:06:59.912856727 +0000 UTC m=+2020.283640517"
Oct 31 01:07:00.251061 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:00.251025    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/2.log"
Oct 31 01:07:00.259850 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:00.259808    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-f6w7r_041328ee-8f09-4927-85e3-f3cceefe3a53/console-operator/3.log"
Oct 31 01:07:01.382185 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:01.382149    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-9cltd_44d79171-f637-4834-bfaa-bd57932eb5b4/volume-data-source-validator/0.log"
Oct 31 01:07:02.686824 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:02.686793    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zmn6t_ef90678c-925d-49bb-acd6-0d2861437916/dns/0.log"
Oct 31 01:07:02.714118 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:02.714091    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zmn6t_ef90678c-925d-49bb-acd6-0d2861437916/kube-rbac-proxy/0.log"
Oct 31 01:07:02.792634 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:02.792606    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbgzq_15983cd6-0227-4191-8770-4fd87b6a52b2/dns-node-resolver/0.log"
Oct 31 01:07:03.533123 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:03.533094    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6hznk_4c7b1d82-f9da-4de8-bb8f-f8e76a938aff/node-ca/0.log"
Oct 31 01:07:04.620140 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:04.620105    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9_2089a27d-9052-4ead-a597-b014675e2de3/istio-proxy/0.log"
Oct 31 01:07:05.914999 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:05.914967    4924 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bd47j/perf-node-gather-daemonset-hpzdv"
Oct 31 01:07:06.076933 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.075684    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/0.log"
Oct 31 01:07:06.076933 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.076549    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/1.log"
Oct 31 01:07:06.135562 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.135529    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-9c7vk_9495ab3a-0b84-4854-bc5e-30d46c9f9040/istio-proxy/0.log"
Oct 31 01:07:06.250616 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.250532    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-p6krc_d0017f59-4050-462e-b9c7-e45a6cf0562e/istio-proxy/0.log"
Oct 31 01:07:06.284134 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.284089    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-pmzpl_fc4b4307-477d-44cd-a66a-66eb109867be/istio-proxy/0.log"
Oct 31 01:07:06.338320 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.338288    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6_95d1f317-798a-49cd-8dbe-1af184a58719/istio-proxy/0.log"
Oct 31 01:07:06.365074 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.365041    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-njw55_ac7e12ef-96ee-46bf-8813-36d45d5e9166/payload-pre-processing/0.log"
Oct 31 01:07:06.397770 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.395031    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:07:06.429335 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:06.429304    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d4b7bcd7-2dvsg_3570c076-c980-454d-9b4d-4fbcc88953e0/router/0.log"
Oct 31 01:07:08.013520 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:08.013490    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22xfm_7db3b75c-d1fb-4b85-bc90-0c9bcbafd9ee/serve-healthcheck-canary/0.log"
Oct 31 01:07:08.690930 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:08.690895    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-gk844_ff9c0bff-850c-4b57-b3fe-ea825f6d4767/insights-operator/0.log"
Oct 31 01:07:08.692149 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:08.692124    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-gk844_ff9c0bff-850c-4b57-b3fe-ea825f6d4767/insights-operator/1.log"
Oct 31 01:07:08.714907 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:08.714874    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bcmv5_a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc/kube-rbac-proxy/0.log"
Oct 31 01:07:08.742064 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:08.742027    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bcmv5_a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc/exporter/0.log"
Oct 31 01:07:08.766698 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:08.766662    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bcmv5_a56b54ef-4dcb-4a3a-9043-1a9a1c31f9bc/extractor/0.log"
Oct 31 01:07:09.015852 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:09.015823    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9_2089a27d-9052-4ead-a597-b014675e2de3/istio-proxy/0.log"
Oct 31 01:07:10.478899 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.478862    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/0.log"
Oct 31 01:07:10.480929 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.480903    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/1.log"
Oct 31 01:07:10.516829 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.516785    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-9c7vk_9495ab3a-0b84-4854-bc5e-30d46c9f9040/istio-proxy/0.log"
Oct 31 01:07:10.600114 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.600085    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-p6krc_d0017f59-4050-462e-b9c7-e45a6cf0562e/istio-proxy/0.log"
Oct 31 01:07:10.628779 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.628717    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-pmzpl_fc4b4307-477d-44cd-a66a-66eb109867be/istio-proxy/0.log"
Oct 31 01:07:10.660296 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.660261    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6_95d1f317-798a-49cd-8dbe-1af184a58719/istio-proxy/0.log"
Oct 31 01:07:10.679501 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.679473    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-njw55_ac7e12ef-96ee-46bf-8813-36d45d5e9166/payload-pre-processing/0.log"
Oct 31 01:07:10.696149 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.696120    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:07:10.717123 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:10.717089    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d4b7bcd7-2dvsg_3570c076-c980-454d-9b4d-4fbcc88953e0/router/0.log"
Oct 31 01:07:11.628732 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:11.628692    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-54459f848c-4w4wn_c25d6a67-6e6f-4703-8bd4-4374035bffbf/manager/0.log"
Oct 31 01:07:11.652063 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:11.652037    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-54459f848c-4w4wn_c25d6a67-6e6f-4703-8bd4-4374035bffbf/copy-manifests/0.log"
Oct 31 01:07:12.595634 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:12.595603    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7556cf8c57nnpb9_2089a27d-9052-4ead-a597-b014675e2de3/istio-proxy/0.log"
Oct 31 01:07:12.946713 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:12.946627    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-55ccbfd5b8-5glzc_db013b71-39dc-4d04-be7a-a4290d785365/manager/0.log"
Oct 31 01:07:13.580815 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:13.580780    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5bb95b6d85-g6jkp_a41765f7-69c2-45e7-bd3e-93bcc31908a1/manager/0.log"
Oct 31 01:07:14.248151 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.248106    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/0.log"
Oct 31 01:07:14.250324 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.250293    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-4f7pj_fc1366be-1d30-4c52-a330-4164ddae0391/istio-proxy/1.log"
Oct 31 01:07:14.284027 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.283992    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-9c7vk_9495ab3a-0b84-4854-bc5e-30d46c9f9040/istio-proxy/0.log"
Oct 31 01:07:14.369205 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.369166    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-p6krc_d0017f59-4050-462e-b9c7-e45a6cf0562e/istio-proxy/0.log"
Oct 31 01:07:14.388576 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.388515    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6f54779779-pmzpl_fc4b4307-477d-44cd-a66a-66eb109867be/istio-proxy/0.log"
Oct 31 01:07:14.426818 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.426788    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-5579cb49c9-29jg6_95d1f317-798a-49cd-8dbe-1af184a58719/istio-proxy/0.log"
Oct 31 01:07:14.442183 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.442027    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-njw55_ac7e12ef-96ee-46bf-8813-36d45d5e9166/payload-pre-processing/0.log"
Oct 31 01:07:14.457389 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.457348    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-fn4lv_bd774ff8-af40-4f55-ac52-cbc18365d9dd/payload-processing/0.log"
Oct 31 01:07:14.477892 ip-10-0-142-217 kubenswrapper[4924]: I1031 01:07:14.477861    4924 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d4b7bcd7-2dvsg_3570c076-c980-454d-9b4d-4fbcc88953e0/router/0.log"